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INTRODUCTION 


This Partnerships Working Group (PWG) study responds to recommendations from the 
National Research Council’s (NRC) NOAA’s Education Program: Review and Critique (2010) 
for NOAA to better understand how NOAA Education partnerships are formed, fostered, 
sustained, and evaluated. The NRC report noted that while partnerships were mentioned 
as a means of achieving NOAA Education strategic goals and objectives, “there is limited 
evidence that current evaluations conducted by NOAA account for the influence of 
partnerships in achieving outcomes and impacts. In addition, most evaluations do not 
attempt to observe the underlying partnership or explore this as a factor in assessment” (p. 
124). The goal of this study is to provide information and recommendations for the NOAA 
Education Council (Council) and NOAA Education community to use in making strategic 
decisions regarding NOAA Education partnerships. 

The PWG was formed to conduct a Partnerships Portfolio Review (PPR). The PWG was 
charged with reviewing and evaluating a portfolio of NOAA Education partnerships. To 
carry out this charge, the PWG reviewed existing documents, revised and implemented a 
survey, and developed case studies. From this analysis, the group identified important 
elements of successful partnerships that can be used to evaluate existing partnerships and 
inform the development of future partnerships. The lessons learned from this review 
inform a set of actionable recommendations for the Council to consider. 

Context and Rationale 

Partnerships are an important strategy in advancing NOAA Education goals and objectives. 
The importance of partnerships is reflected in both the NRC report (2010) and the NOAA 
Education Implementation Plan (2010). 

Limited education resources and the inherently global nature of NOAA’s mission 
make strategic partnerships necessary in order for the agency to accomplish its 
ambitious goals. (NRC 2010, p. 3) 

In the partnership and collaboration section, NOAA states that it is the leading 
science and service agency in ocean and atmospheric science, and thus it has the 
responsibility to increase its role as a coordinator and collaborator in these areas of 
science education. The America COMPETES Act is cited as a mandate to serve as a 
“catalyst” to strengthen oceanic and atmospheric science. The strategic plan points 
to a broad array of potential partners, including other agencies, businesses, 
organizations, professional societies, education associations, and school systems. 

All partnerships will be developed in the interest of the agency using its resources 
to “advance the environmental literacy and scientific knowledge of our Nation and 
the global community.” Partnerships will be critical if NOAA is to reach the 
ambitious goals identified in the strategic plan, because the agency does not have 
the resources to achieve its goals on its own. (NRC 2010, p.60) 

Innovation in content delivery is also empowered by participation in strategic 
partnerships and organizational networks with formal and informal institutions 
that share NOAA’s educational goals. Fostering these relationships allows the 
agency to leverage its resources to connect with the public through local and 
regional informal science venues; without such partnerships, these connections 
would otherwise be cost prohibitive. (NOAA Education Implementation Plan, 2010) 
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The NRC publication also recognized a lack of direction regarding partnerships: “Although 
the importance of partnerships is stressed in the [NOAA Education Strategic] plan, there is 
no specific guidance about how or with whom to partner” (NRC, 2010, p.5). Accordingly, 
the NRC publication includes a recommendation that NOAA should “evaluate where it 
leads, collaborates, follows or declines to participate in partnerships with others” (NRC, 
2010, p.137). 

Evaluation of such partnerships can also help inform the revisions of the NOAA Education 
Strategic Plan and Implementation Plan. To further investigate NOAA Education 
partnerships, the PWG was established. 

Background 

In 2010, the Council organized a set of working groups to implement the NOAA Education 
Strategic Plan. Two of the working groups were the Finding Innovative Ways to Connect 
NOAA to the Public working group (Connects) and the Pre-Kindergarten through Early 
Career (PK-20) working group. 

The Connects working group’s tasks focused on informal education and included a survey 
of all NOAA Education informal partnerships. The PK-20 working group’s tasks focused on 
formal education and was in need of a partnership survey. The Council advised the two 
working groups to develop and disseminate the survey together to meet the following 
objectives: 

• Determine the types of partnerships NOAA Education programs participated in; 

• Identify overlaps and gaps based on NOAA Education strategic plan needs; and 

• Determine which partnerships demonstrate the best return. 

In 2012, the Council approved the PWG, which focused on completing an evaluation of the 
NOAA Education partnership portfolio. The PWG leads were Frank Niepold and Kate 
Thompson. Dr. Diana Payne of Connecticut Sea Grant served as the Evaluation Lead, 
working in direct collaboration with Dr. John Baek. PWG members included: Julie Bursek, 
Ron Gird, Maria Murray, Seaberry Nachbar, Frank Niepold, Rochelle Plutchak, Shannon 
Rides, and Sarah Schoedinger. The PWG was tasked with: 

• Completing and disseminating the partnerships survey to better understand the 
nature of existing NOAA Education partnerships; 

• Conducting an evaluation study to identify and better understand the nature of 
successful partnerships; 

• Developing a set of actionable recommendations to present to the Council; and 

• Sharing findings with the Council. 
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METHODS 


The PWG conducted the evaluation study from January 2013 to February 2014. The 
evaluation approach was a participatory process, focusing on the involvement of staff from 
multiple programs in the evaluation of NOAA Education partnerships with the support of 
professional evaluators. This approach is called a portfolio review. 

Study Design 


Portfolio Review 

A portfolio review is an internal evaluation, conducted by a team of agency staff, of a focal 
area of an agency’s portfolio of efforts in relation to goals or outcomes (Yee, 2012). Several 
steps encompass a portfolio review: determining study scope, developing logic models, 
developing research questions, conducting a data inventory, collecting and analyzing data, 
selecting evaluative criteria, developing actionable recommendations, and drafting a final 
report. All steps of the PPR process were conducted by PWG members with support from 
evaluation leads. 

Scope 

The scope of an evaluation study defines the subject matter assessed, including a program 
or aspect of a program and the time periods and locations that will be included (GAO, 
2012). The study was inclusive of active and inactive external partnerships undertaken by 
NOAA Education professionals from programs serving on the Council. 

The scope was informed by a similar study conducted by the Smithsonian Institution Office 
of Policy and Analysis (2012) examining the portfolio of partnerships across all 
Smithsonian units. The study conducted a “baseline inventory of unit active partnerships 
with external organizations” (p. 7) via an online partnership survey instrument. The report 
summary included a recommendation to develop of case studies, in part to “discover what 
partnerships are successful and why, whether they can be expanded, and whether they 
can provide some variant of lessons learned.” (p. 35) 

The portfolio of NOAA Education partnerships may number in the hundreds, and the PWG 
did not have the three years it took Smithsonian to conduct an exhaustive and 
comprehensive study of its many partnerships. To focus the study, the PWG decided on a 
subset of the portfolio deemed high-return partnerships (see p.8). PWG members identified 
the need to highlight the unique mechanisms and value of NOAA Education partnerships 
in the current budget climate. 

Additionally, the PWG identified the outcomes of the study and how study results might be 
implemented. The deliverables of the PPR included the following: 

1. A sampling of NOAA Education high-return partnerships, including leveraged 
dollars, gaps, and overlap; 

2. A model of how to initiate and manage effective, high-return partnerships; and 

3. A final report to communicate the NOAA Education partnership portfolio to 
leadership and stakeholders. 

The PWG intended the results of the PPR to inform potential future working group 
activities, including: 
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1. Develop a partnerships performance measure; 

2. Develop a training manual; 

3. Inform strategic planning; 

4. Provide feedback on progress toward the NRC recommendations; and 

5. Develop a standardized partnership survey instrument. 

Research Questions 

The research questions of an evaluation study are constructed so that the issues and 
concerns of a program’s stakeholders about program performance can be articulated and 
focus the evaluation to help ensure that its findings are useful (GAO, 2012). In addition to 
recommendations made by the NRC, the following questions were developed by the PWG 
leads and evaluation leads at an initial meeting in 2013 and reviewed by the PWG 
members: 


1. How can NOAA partners help complete our mission? 

2. In what ways does NOAA maximize its Education partnerships? 

3. What commonalities are shared by NOAA Education high-return partnerships? 

4. What tools and strategies are needed to foster an effective NOAA Education 
partnership portfolio? 

Evaluation Design 

Based on the research questions, the PWG leads and evaluation leads outlined the 
following sequential, mixed-methods design to help answer the research questions (Figure 
1). The results of the quantitative component (e.g., survey) would provide a broad view of 
the portfolio. The qualitative component (e.g., case studies) would examine partnerships 
by analyzing mechanisms and outcomes related to success. 

Figure 1. NOAA Education Partnerships Portfolio Review evaluation design. 
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Timeline 

The PWG leads and Evaluation leads met to discuss initial goals and drafted a timeline. The 
final timeline (Figure 2) deviated from the original plan due to unforeseen delays (i.e., 
October 2013 government shutdown). 


Figure 2. NOAA Education Partnerships Portfolio Review Timeline. 



Data Collection & Analysis 

Two types of evaluation activities were conducted to help understand the nature of NOAA 
Education high-return partnerships: an online survey and case study examples resulting 
from the survey data. 

Survey Design 

Based on previous work of the PK-20 WG and Connects WG, the PWG reviewed an existing 
NOAA Education Partnership draft survey and then defined the partnerships and 
assessment criteria. Note that not all criteria need to be met in assessing and/or selecting a 
partner. The PWG gathered and reviewed the following sources of evidence: 

• MOU/MOA of existing NOAA Education partnerships; 

• NOAA Education Partnership draft survey; 

• Geographic database of NOAA Education programs and facilities; 

• NRC Report: NOAA’s Education Program: Review and Critique (2010); 

• NOAA Education Strategic Plan 2009-2029 (2009); 

• NOAA Education Implementation Plan (2010); 

• Rhode Island Sea Grant Partnership Principles; and 

• Smithsonian Institution partnerships report (2012). 

The PWG established the following definitions and criteria for the survey. 

• Partnership - An agreement between two or more organizations, created to achieve 
or assist in reaching a common goal. Partnerships may involve one organization 
utilizing another's unique abilities, equipment or services, or it may be a "sharing" 
of resources (e.g., money, time, knowledge, equipment) to accomplish shared 
objectives between the participating partners. 
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• High-return Partnership - Benefits to NOAA and partners outweigh the investment 
and involvement for all parties involved. 

Criteria for assessing a partnership: 

• The partner shares similar values and qualities to those of NOAA. Our corporate 
values are Science, Stewardship and Service. Our agency is successful, welcoming, 
trusted, innovative, reliable, customer-focused, and collaborative; 

• The partnership creates stories or case studies that will be relevant, interesting and 
intriguing for NOAA stakeholders directly as a result of operating unit(s) 
involvement; 

• The partnership provides opportunities for NOAA involvement by either our staff 
or our constituents; 

• The partnership provides a high return to NOAA’s local communities where NOAA 
offices/labs/sanctuaries and reserves are located (high return means benefits to 
NOAA and partners outweigh the investment and involvement for all parties 
involved); 

• The partnership provides a high return to NOAA staff and our constituents (high 
return means benefits to NOAA and partners outweigh the investment and 
involvement for all parties involved); 

• The partnership provides a high return to International/National/regional/local 
initiatives (high return means benefits to NOAA and partners outweigh the 
investment and involvement for all parties involved); 

• The partner organization has broad connections with communities of influence 
that would further our mission and goals; 

• The partnership includes opportunities for NOAA (e.g., conservation, stewardship, 
science, awareness, safety and preparedness); 

• The partnership is designed to make a lasting contribution to NOAA communities; 
and 

• The partner is helping us address our education strategic goal/objectives. 

Criteria for selecting a partner: 

• Partner has established relationships that will help provide a different level of 
policy access to those networks for NOAA Education officials; 

• Partner has established relationships that will help provide a different level of 
science content knowledge (technical expertise) to those networks for NOAA 
Education officials; 

• Partner has established relationships that will help NOAA provide a different level 
of access to new or refined approaches for audiences, constituents, stakeholders; 

• Partner can contribute to NOAA’S workforce development goals; and 

• Partner is selected through competition. 

Survey Administration 

The NOAA Education Partnership Survey (Appendix A) was distributed to members of the 
Council and their education networks in late spring/early summer 2013 with a close date of 
July 1, 2013. A total of 67 partnerships were identified in the survey responses. It is 
important to note that this was not a comprehensive survey, and some programs did not 
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respond to the survey at all. Each Council member could provide up to five examples of 
high-return partnerships. 

Survey Data Analysis 

Fourteen analysis questions were developed by evaluation leads to help guide working 
group members in analyzing the survey data. The questions were developed based on the 
Partnership Survey items, survey responses and evaluation questions. The answers 
provided a broader understanding of who NOAA Education partners with, the strategies 
for partnering, and the resulting outcomes of the partnership. The 14 questions were: 

1. With whom does NOAA Education partner? 

2. What types of institutions are represented in NOAA Education partnerships? 

3. How does NOAA Education formalize partnerships? 

4. What is the current status of NOAA Education partnerships? 

5. How long have NOAA Education partnerships been in place? 

6. Where are NOAA Education partners? 

7. How much funding has NOAA contributed to the partnership? 

8. How much staff time has NOAA contributed to the partnership? 

9. Does the partner share NOAA Education goals and objectives? 

10. Does the partner align with NOAA Education output measures? 

11. Does the partnership allow NOAA to reach new audiences? 

12. Does the partnership provide access to expertise, facilities and technology NOAA 
does not have? 

13. Does the partnership allow NOAA to provide unique expertise and/or 
opportunities? 

14. Could the outputs and activities be achieved in the absence of the partnership? 

PWG members volunteered to investigate one or two questions each and were the primary 
analyzers of the data with support from the evaluators. PWG members shared and 
reviewed results with the evaluation leads. Most of the analysis involved qualitative coding 
by identifying natural categories in the survey responses. Results are presented in 
percentages to provide an understanding of part-whole relationships and an overall 
understanding of the portfolio of high-return partnerships. 

Case Studies 

According to the U.S. Government Accountability Office (GAO, 2012), “If the objective of the 
case study is to describe aspects of an issue, provide context, or illustrate findings 
developed from a more broadly applied survey, then selecting a non-generalizable sample 
of cases may be appropriate” (p.34). The PWG leads and Evaluation leads selected the case 
study as an appropriate methodology to better understand the nature of NOAA Education 
high-return partnerships. Case studies “can be used to document and analyze the outcomes 
of interventions...[including] programs sponsored by federal agencies” (Yin, 2012, p. xix), 
and are useful when considering an initiative that takes place in a “real-world setting” (Yin, 
2012), to illustrate findings, or present an in-depth analysis (GAO, 1990). Because each of 
these aspects was important in the analyses, the PWG reviewed the NOAA Education 
Partnership Survey dataset to determine which of the reported partnerships might best 
illustrate high-return. 

The PWG identified case studies that represented broad categories of partners identified in 
the survey data and to which working group members could easily access data. The PWG 
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had limited time and needed to rely on the working group to document partnership case 
studies (i.e., convenience sampling). With examples of case study partnerships selected, 
three broad categories were identified as core models based on the type of partner: 1) 
informal education institutions; 2) nonprofit organizations; and 3) partnerships with a 
local focus. 

Once examples were suggested and discussed for each core model, PWG members 
volunteered to write up a case study. PWG members utilized data collected in the survey 
and their own records and documents of the partnerships, and consulted with partners to 
gather additional information. Authors were provided with a template to guide writing up 
the case study to ensure that necessary information was included. 

Logic Model 

Logic modeling is an evaluation technique encouraging careful consideration of the 
relationships between program activities and results (Knowlton & Phillips, 2009). Program 
logic models in diagram format are visual depictions of how an organization does its work. 
“At a minimum, a program logic model should outline the program’s inputs, activities or 
processes, outputs, and both short-term and long-term outcomes—that is, the ultimate 
social, environmental, or other benefits envisioned” (GAO, 2012, pp.10-11). The PWG 
developed the logic model, fully described in the Discussion section of this report. The logic 
model describes how NOAA Education partnerships lead to the outcomes of increased 
reach, efficiency and effectiveness. 

Final Presentation and Report 

The Evaluation leads reviewed data analyses and case study drafts to begin development of 
the final report structure. Reviews of the final report and the development of actionable 
recommendations were conducted by the PWG in January 2014, with a presentation to the 
Council in February 2014. The draft was updated and revised based on comments made at 
the presentation, with final delivery of the report to the Council in July 2014. 
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RESULTS 


Survey Data 

A total of 14 Analysis Questions were answered based on the survey data. The questions 
were developed by the evaluation leads based on the original NOAA Education Partnership 
Survey items, survey responses and research questions. 

The answers provide a broader understanding of whom NOAA Education partners with, 
the strategies for partnering, and the resulting outcomes of the partnership. Because the 
response to the survey was incomplete, readers should not generalize to the whole 
universe of partnerships that exist within the NOAA Education community. Additionally, 
because respondents were able to indicate multiple answers or chose not to answer 
questions, the number of answers (n) differs between questions. 

Analysis Question 1: With whom does NOAA Education partner? 

A total of 67 NOAA Education partnerships were identified in the survey. The partners are 
presented in Table 1. Some partner organizations were listed multiple times because they 
were partners in different partnerships. 


Table 1. NOAA Education Partners. 


Ala Kama Foundation 

National Marine Sanctuary Foundation 

American Meteorological Society (AMS) 

National Park Service, Great Lakes Research and 

Annapolis Maritime Museum 

Education Center 

Ao"ao 0 Na Loko I'a 0 Maui Fishpond 

National Science Teachers Association 

Association 

Navarre Beach Marine Science Station (NBMSS) 

Aquarium at Moody Gardens 

New England Aquarium 

Aquarium of Niagara, Buffalo Museum of Science 

Oakland Museum of California (OMCA) 

Awaiaulu 

Ocean Discovery Institute 

Bishop Museum 

Oregon Coast Aquarium (OCA) 

California Academy of Sciences (CAS) 

Pacific Tsunami Museum 

Channel Islands National Park (CINP) 

Pennsylvania Sea Grant 

City and County of Honolulu 

PLAN IT NOW (PIN) 

City College of the City University of New York 

Randall Museum 

Coral Reef Alliance 

Science on a Sphere (SOS) 

Council of State Science Supervisors 

Seacoast Science Center 

Exploratorium 

Seattle Aquarium 

Farallones Marine Sanctuary Association 

Smithsonian Institution, National Museum of 

Feiro Marine Life Center 

Natural History 

Florida A&M University 

South Carolina Aquarium 

Florida Department of Environmental Protection, 

Texas A&M University - Galveston 

Office of Environmental Education (FL DEP OEE) 

The Mariners’ Museum 

Galveston Island Nature Tourism Council 

The Weather Channel 

Georgia Aquarium 

Turtle Bay Resort 

Georgia Southern Museum 

Tybee Island Marine Science Center 

Great Lakes Aquarium 

U.S. Forest Service 

Gulf of Mexico Foundation 

UAF School of Fisheries and Ocean Sciences 

Howard University 

University of Georgia Marine Extension Service 

Huron-Clinton Metropolitan Authority 

University of Maine Cooperative Extension 

Johns Hopkins Center for a Livable Future 

University of Maryland Eastern Shore 

MADE CLEAR (Maryland and Delaware Climate 

University of Rhode Island 

Change Education Assessment and Research) 

USFWS Kilauea Point National Wildlife Refuge 

National Association of Black Scuba Divers 

Waikiki Aquarium 

National Earth Science Teachers Association 

Western Pacific Coral Reef Institute (WPCRI) 

National Environmental Education Foundation 

Whaler's Village/Oceanology 
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Analysis Question 2: What types of institutions are represented in NOAA Education 
partnerships? 

NOAA partners with a wide variety of organizations. The resulting number in this question 
is greater than the number of partnerships represented in the survey data because in some 
cases the partnership is among NOAA and several different types of organizations. 

Of the 67 NOAA education partnerships described in the survey, at least one partnership 
(the Science on a Sphere® Collaborative Network) comprises several different types of 
institutions and organizations, so percentages in this question are calculated based on 71 
institutions. Zoos, aquariums and museums (ZAMs) make up the largest portion (32%, 
n=23) of institution types, with nonprofit professional and nongovernmental organizations 
representing 25% (n=18), and colleges and universities representing 17% (n=12) of all 
partnering institution types reported (Figure 3). 


Figure 3. Type of Partner Institutions 
ZAMs 
Nonprofits 
Higher Ed 
Federal Agency 
Local Agency 
Corporation 
K-12 
State Agency 
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Analysis Question 3: How does NOAA Education formalize partnerships? 

Most NOAA Education partnerships are formalized at some point in during their existence, 
although multiple mechanisms may be used throughout the life of the partnership. 
Respondents were given the option of grants, cooperative agreements, memoranda of 
agreement (MOA)/memoranda of understanding (MOU), contracts. Thus the number of 
responses is greater than the 67 partnerships represented in the survey. 

Of the 83 mechanisms identified in the survey, greater than 50% (n=46) are funded through 
a grant or cooperative agreement at some point during the tenure of the partnership 
(Figure 4). Almost a quarter (24%; n=20) have been articulated through MOA’s or MOU’s. 
These instruments may co-occur with grants and cooperative agreements, but not 
exclusively so. Only three (15%) of the MOA’s reported involved a partnership with a local, 
state or federal agency. Almost 20% (n=16) of the surveyed partnerships have no formal 
mechanism articulating the partnership. In these cases, the support is provided in-kind. 

Figure 4. Type of Partnership Mechanisms. 


Analysis Question 4: What is the current status of NOAA Education partnerships? 

To describe the level of activity and operation of the partnership, respondents could select 
from following four options: 1) active and fully operational (76% to 100%), 2) partially 
operating (51% to 75%), 3) minimally operating (1 to 50%), 4) not active (0%). A total of 59 
responses were received for this item, which did not require a response. As depicted in 
Figure 5, 85% (n=50) of NOAA Education partnerships identified in the survey are active 
and fully operational, while 10% (n=6) of NOAA partnerships are partially operating. Two 
respondents (3%) selected minimally operating, and only one respondent (2%) indicated 
that the partnership is not active. 

Figure 5. Current Status of the Partnership. 
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Analysis Question 5: How long have NOAA Education partnerships been in place? 

Respondents were asked to provide the year the partnership began. A total of 64 responses 
were received for this question, which did not require a response. More than three-fourths 
of the partnerships were new or at mid-point (compared to all responses received), having 
been in place for 11 years or less. Most NOAA Education partnerships identified in the 
survey began within the last three years (2009-2012; n=18, 28%), although the four-to-seven 
(n=16, 25%) and eight-to-eleven (n=15; 23%) year ranges also rated high (Figure 6). Some 
respondents indicated that partnerships were in place for many years, but became formal 
partnerships through an MOU/MOA within the last seven years. 


Figure 6. Length of 

I 

0 to 3 years 
4 to 7 years 
8 to 11 years 
12 to 15 years 
20 to 22 years 
16 to 21 years 
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Analysis Question 6: Where are NOAA Education partners? 

Based on 67 partnership descriptions, five categories were created to group the responses 
based on the geographic focus of the partnership: local, state, regional, national, and 
international. Local is defined as an area smaller than a large portion of a state. The 
definition for state covers most or all of a state. Regional encompasses at least two states in 
the same geographic area. National includes many if not all states, geographically spread 
across the country. International indicates at least two countries. As depicted in Figure 7, 
local was the largest percentage (40%, n=27) of partnerships represented in survey. The 
next largest group was national (24%, n=16), followed by state (21%, n=14) and regional 
(12%, n=8). Only two (3%) partnerships were reported as international: PLAN !T NOW and 
the Science On a Sphere® Users Collaborative Network. 

Figure 7. Geographic Focus of the Partnership. 



Respondents were asked to provide the city, state and zip code of the partner. In Figure 8, 
red pins represent partner locations as specified in the 2013 NOAA Education Partnerships 
survey. Yellow dots represent locations of NOAA offices, labs, sanctuaries, and reserves. 

Figure 8. NOAA and Partner Locations. 



Note: Image provided courtesy of Ronald Tardiff. 
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Analysis Question 7: How much funding has NOAA contributed to the partnership? 

Survey respondents were asked to provide the total contribution in US dollars, which 
included the monetary value of staff time, other in-kind contributions, and cash. Based on 
these data, an estimate of how much NOAA provided in funding in one year to the partners 
was $17,021,262. NOAA partners with a variety of organizations, with levels of funding that 
can vary from zero dollars to over one million dollars. For the majority of partnerships, 
NOAA provided less than 10 thousand dollars of funding (36%, n=23), as shown in Figure 9. 
The majority of funding to partners has been via multi-year grants with universities (60%, 
n=4) and non-profit organizations (30%, n=3). 


Figure 9. NOAA Funding Contributed to the Partnership. 


$0 to 10k 
$1 Ok to 100k 
$100k to 1M 
> SIM 



Analysis Question 8: How much staff time has NOAA contributed to the partnership? 

Based on the 64 responses to this survey question, which did not require a response, FTE 
estimates were made based on verbal descriptions or when multiyear estimates were 
provided. NOAA contributed a total of 23 full-time equivalents (FTEs) to Education 
partnerships. The partnerships that have historically received the most FTE support have 
been university and non-profit partnerships which match almost directly with NOAA 
funding support. Figure 10 depicts the breakdown across range of staff time contributions. 
The majority (77%, n=49) of partnerships are maintained with less than 0.5 FTE, as shown 
in Figure 10. 


Figure 10. NOAA FTE Contributed to the Partnership. 


0 to 0 1 
0 1 to 0 5 
0 5 to 1 0 
> 1 0 
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Analysis Question 9: Does the partner share NOAA Education goals and objectives? 

In terms of criteria for selecting a partner, each respondent was able to indicate multiple 
selections based on NOAA Education goals and objectives (n=116). As represented in Figure 
11, 38% (n=44) of the partnerships were established to help provide increased access to 
new or refined approaches for audiences, constituents and, stakeholders. Additional 
responses included providing increased level of science content knowledge or technical 
expertise (24%, n=28), contributing to NOAA’s workforce development goals (10%, n=12), 
selecting the partner through a competitive process (10%, n=12), and helping provide a 
different level of policy access (8%, n=9). Some respondents (10%, n=12) elaborated by 
selecting other. 


Figure 11. Criteria for Selecting Partners. 
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Analysis Question 10: Does the partner align with NOAA Education output measures? 

In assessing the partnership in terms of alignment with NOAA educational output 
measures, each respondent was able to indicate multiple measures (n=131). For example, a 
partnership might include both formal education and professional development program 
activities. As illustrated in Figure 12, 31% (n=41) indicated the partnership aligned with 
informal education programs, 27% (n=36) with formal education programs, 24% (n=32) 
with professional development programs, and 9% (n=12) with higher education programs, 
while 8% (n=10) were not specified. 


Figure 12. Alignment with NOAA Education Output Measures. 
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Analysis Question 11: Does the partnership allow NOAA to reach new audiences? 

To answer these questions, the survey responses were reviewed and synthesized into eight 
categorical bins. Sixty-nine descriptions of new audiences reached were identified. Overall, 
new audiences were reached by 97% (n=67) of the partnerships reported. Of these new 
audiences, the largest group indicated was visitors to an institution in a new geographic 
area (30%, n= 20), followed by visitors to a new institution with access to a higher capacity 
(23%, n=16) as shown in Figure 13. Other audiences indicated included access to a 
traditional or native community (11%, n=8), access to students due to higher capacity (9%, 
n=6), minority students and underserved populations (8%, n=5), access to new volunteers 
(7%, n=4), and science teachers (6%, n=3) and “other” (6%, n=3). The category “other” 
included graduate and undergraduate students, general public, docents, and families. 


Figure 13. Reaching New Audiences 
Geographic Area 
Higher Capacity 
Native Community 
Students 
Minority 
Volunteers 
Science Teachers 
Other 



Analysis Question 12: Does the partnership provide access to expertise, facilities and 
technology NOAA does not have? 

Many partners indicated providing more than one type of contribution, thus the higher 
number of total contributions identified (n=84). Contributions were grouped as technology 
(45%, n=38), facilities (31%, n=26) and expertise (24%, n=20), as depicted in Figure 14. 


Figure 14. Partner Contribution to the Partnership. 
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Analysis Question 13: Does the partnership allow NOAA to provide unique expertise 
and/or opportunities? 

Survey results indicate that NOAA provides technical and knowledge transfer. Of the 67 
responses depicted in Figure 15, 94% (n=63) indicated that NOAA provided expertise, either 
scientific or educational. The NOAA materials category was identified by 67% (n=45), and 
ranges from handouts at special events at partnership institutions to curriculum 
collections. Physical space (e.g., facilities, exhibit space, real estate) was identified by 36% 
(n=24). Other resources identified by respondents include exhibit development (37%, n=25) 
and instruments and lab equipment (18%, n=12). Some open-ended responses mentioned 
opportunities for students to work alongside NOAA scientists. 


Figure 15. NOAA Expertise/Opportunities Contributed to the Partnership. 
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Analysis Question 14: Could the outputs and activities be achieved in the absence of 
the partnership? 

When considering NOAA Education partnerships, the PWG also contemplated what the 
partnership added in value and tried to consider what benefits were caused from the 
partnership or may not have happened in the absence of the partnership. This, however, 
was difficult to ascertain given the scope of the study. 

Qualitative data provided the direct response this question. Specific comments evoked by 
this question included: 

• We could not deliver the programs on which we collaborate without this 
partnership. 

• We have one part-time education staff person and cannot achieve our desired 
outcomes without significant partnerships. 

Quantitatively, respondents were requested to assess the partnership based on criteria to 
indicate outputs and activities the partnership provides. Of the 67 responses shown in 
Figure 16, opportunities for NOAA messaging rated highest (84%, n=56). Partnerships 
provided opportunities for NOAA involvement by either staff or constituents (75%, n=50), 
provided a high return to NOAA’s local communities where NOAA offices, labs, sanctuaries 
and reserves are located (64%, n=43), and provided opportunities for NOAA involvement 
by either staff or constituents (60%, n=40). Many respondents indicated the partnership 
created stories or case studies that will be relevant, interesting and intriguing (46%, n=31), 
and provides a high return to international, national, regional, and local initiatives (46%, 
n=31). 


Figure 16. Partnership Assessment Criteria. 
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Core Models 


PWG members recommended potential case studies, reviewed the options, and selected 
several examples. The PWG then identified thematic core models for the case studies, 
based on the type of partner: 1) informal education institutions, 2) non-profit institutions, 
and 3) partnerships with a focus on local issues. 

The PWG selected partnerships with informal education institutions (i.e., zoos, aquariums 
and museums) and with non-profit/non-governmental organizations (i.e., policy or 
professional organizations), as these were most often reported in the NOAA Education 
Partnerships Survey (32% and 25% respectively). Partnerships with a local focus were 
selected to highlight the reach of NOAA. As noted by the NRC (2010), “[i]ndividual offices 
have separate mandates and often have local components with local control.” (p. 143). 

Core models serve as cross-case synthesis and were recommended by the Evaluation 
Co-Leads, as the analysis of case studies can occur across multiple cases to synthesize 
findings from individual cases and identify patterns (Yin, 2012). The three core models are 
described below, with specific examples provided in the following section. 

Core Model 1: Partnerships with Informal Education Institutions 

NOAA Education programs have numerous partnerships with informal education 
institutions (e.g., zoos, aquariums, museums). Respondents to the NOAA Education 
Partnerships survey indicated 32% of partnerships reported were of this type. Partnerships 
with informal education institutions help NOAA: 1) increase access and reach to broader 
audiences and diverse populations; 2) better distribute NOAA science, data, technology and 
issues important to NOAA (e.g., via exhibits, websites); and 3) facilitate broader impacts of 
NOAA science and scientists. A key component of successful high-return partnerships with 
informal education institutions is communication between specific points of contact for 
each partner. The contacts are committed to sustaining the partnership, and may or may 
not have been integral to the formation of the partnership. The mutual points of contact 
can also facilitate opportunistic collaborations to capitalize on the capabilities each partner 
brings to the relationship. Examples of NOAA Education high-return partnerships with 
informal education institutions are noted below and more fully described in the following 
section. 

• Case Study 1A: NOAA and the Exploratorium 

• Case Study IB: NOAA Office of National Marine Sanctuaries (ONMS) and the 
Oakland Museum of California (OMCA) 

• Case Study 1C: NOAA and Science on a Sphere® (SOS) 

Core Model 2: Partnerships with Nonprofit Institutions 

Partnerships with nonprofit institutions were indicated in 25% of the responses to the 
NOAA Education Partnerships survey. NOAA Education partnerships with nonprofit 
organizations are generally more targeted than partnerships with informal education 
institutions in terms of audience (e.g., inservice teachers, children) and content (e.g., 
atmospheric and ocean science; surviving severe weather events). For NOAA Education, 
such specific endeavors have resulted in 1) long-term involvement of partners and 
participants; 2) a product that has been leverageable for additional support (funding 
and/or in kind) from other federal agencies and partners; and 3) involvement of highly 
recognized celebrities in the support and promotion of projects. Key components to success 
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include meaningful evaluation and support from program leadership. Two exemplary 
case studies are noted below. 

• Case Study 2A: NOAA and the American Meteorological Society (AMS) 

• Case Study 2B: National Weather Service (NWS) and PLANIT NOW 

Core Model 3: Partnerships with a Focus on Local Issues 

Given the geographic and content diversity (e.g., ocean, coast, Great Lakes, atmosphere, 
weather) of NOAA Education programs as well as the individual education mandates 
within specific programs, forging partnerships with a focus on local issues is a unique 
strength of NOAA. Partnerships with a local focus highlight the reach of NOAA, and survey 
respondents indicated that 61% of the partnerships take place at the local or state level 
(40% local, 21% state). Such partnerships allow NOAA to: 1) capitalize on interest in local 
issues directly connected to agency mission and goals; 2) encourage local and often 
sustainable collaboration among scientists, educators, volunteers and others on common 
community issues; and 3) leverage local and/or regional funding and resources (e.g., 
volunteers). Key components for success include established personal relationships - the 
“people factor” - between key individuals knowledgeable about the local issues and the 
leveraging of in-kind support of resources. The following two case studies demonstrate the 
capacity and effectiveness of partnerships focused on local issues. 

• Case Study 3A: EPA Great Lakes National Program Office (GLNPO) and the Center 
for Great Lakes Literacy (CGLL) 

• Case Study 3B: Channel Islands National Marine Sanctuary (CINMS) and Channel 
Islands National Park (CINP) 


22 



Case Study 1A: NOAA and the Exploratorium 

A partnership initiated in 2009 between NOAA and the Exploratorium connects NOAA 
scientists and science content with an informal education institution serving hundreds of 
thousands of people annually via its physical space and online presence. 

Background 

Located in San Francisco, California and founded in 1969, the Exploratorium is an 
exceptional hands-on public science center consisting of approximately 600 physical 
exhibits offering experiential learning experiences difficult to obtain through any other 
medium. The Exploratorium has also expanded its reach beyond its own doors by sharing 
exhibits with other museums globally and through a strong web presence. Its staff and 
programs are leaders in the field of exhibit design and best practices training, and have 
had a significant impact on the worldwide museum and science center community. 

According to 2012 self-reported statistics, the Exploratorium’s reach includes the following: 

• 570 thousand people visited the Exploratorium (55% adults; 45% children); 

• 52% of the visitors were from the Bay Area, 24% from the rest of California, 14% 
from other states, and 10% from outside U.S.; 

• 115,000 students and chaperones; 

• 36% of visitors received free or discounted admission; 

• 44 thousand attended on “Free Wednesdays”; 

• 180 million visited Exploratorium exhibits at other science centers and locations 
worldwide; 

• 13 million people visited www.exploratorium.edu and viewed the 50 thousand 
pages of original content; 

• 75 live webcasts, podcasts, and videos are produced; and 

• New over-the-water location on San Francisco Bay may bring in an estimated 1 
million ticketed visitors per year. 

In 2009, NOAA and the Exploratorium signed a 5-year MOU, which described a 
multi-faceted and in-depth collaboration between the two institutions. As the 
Exploratorium moved to a new location on San Francisco Bay, an expansion into 
environmental science content was also planned. Both NOAA and the Exploratorium were 
well positioned to undertake the challenge of creating a citizenry engaged in current 
environmental issues. 

The partnership covers a broad range of activities, including the co-development of 
exhibits, education and public programs, ship docking, media-rich websites, staff 
exchanges, and learning research programs focused on critical issues important to NOAA. 
The goal is to promote public engagement to help society better understand NOAA science, 
global environmental change and the ways in which individuals can create a healthier, 
sustainable future for the planet. 

The collaboration leverages each partner’s strengths in translating accurate, up-to-date 
science content and understanding of complex systems into learning opportunities for the 
public. The effort also benefits the fields of informal science education (ISE) and scientific 
research by exploring new methods of engaging audiences regarding critical and dynamic 
environmental sciences. 
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While some of the partnership work was supported by NOAA Environmental Literacy 
Grants (ELG) program, particularly the Scientist-in-Residence Program, the majority of the 
work and support for the partnership came from in-kind contributions authorized by the 
formal MOU. The MOU was flexible and allowed for experimentation and engagement 
between NOAA and the Exploratorium. The MOU engaged all aspects of NOAA in the 
partnership. 

The Exploratorium works on a regular basis with Carrie McDougall, NOAA Office of 
Education (OED), Rochelle Plutchak (OAR), and Maria Murray (OED). There is a standing 
call every two weeks with counterparts at the Exploratorium, Mary Miller and Kate 
O’Donnell. This group does most of the planning and high-level work, but the partnership 
is designed so that this group does not become a bottleneck. 

The partnership is governed by an eight-member Vision Council that consists of four 
members each from NOAA and the Exploratorium. One member from each is rotational, 
based on the projects underway at the time. The Vision Council meets biannually. 
Traditionally, the summer meeting has been at a NOAA location and paired with a site 
visit, while the winter meeting has been in San Francisco concurrent with the American 
Geophysical Union meeting in December. This capitalizes on opportunities to involve 
scientists in conversations with the Exploratorium. 

The Exploratorium has visited and had in-depth conversations with offices across NOAA, 
including, but not limited to, OAR labs, NMFS science centers, NOS National Marine 
Sanctuaries, NWS local forecast offices and CPC, and NESDIS NCDC and NSOF. Many strong 
relationships have been built as a result of these interactions, and the Exploratorium feels 
that they can reach out to their contacts directly when needed. 

Results 

Numerous activities have resulted from the partnership between NOAA and the 
Exploratorium: 

• Scientists-in-Residence - The Scientist-in-Residence grant, awarded through the 
OED-managed Environmental Literacy Grants program, enabled four groups of 
NOAA scientists to participate in week- and month-long residences on the 
Exploratorium floor. Working with Exploratorium “Youth Explainers”, NOAA 
scientists strengthened their skills in communicating their research to the public, 
and shared NOAA scientific expertise for use in Exploratorium exhibit mediation 
and development. 

• Scientific instruments - The Exploratorium’s new waterfront location offers an 
opportunity to engage the public in environmental science as well as collect data 
that can be shared with both the public and the larger science community. The 
Exploratorium equipped the pier with sensors to gather data on the water, 
atmosphere, weather, and biological communities of the San Francisco Bay, 
including a NOAA buoy to measure aspects of ocean acidification. Recently, the 
Exploratorium added an HF Radar installation, which reports to CENCOOS. The 
Exploratorium is also now an institutional member of CENCOOS with 
representation on the governing board, and will become a data collection site for 
the PORTS network. 

• Exhibit development - NOAA scientists are intensively involved in the conception, 
development and completion of new Exploratorium exhibits. 
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• Ship docking - The Exploratorium’s location on the pier offers a prime location for 
NOAA (and other) research vessels to tie up during routine operations. Several 
successful NOAA ship dockings have occurred since the new location opened in 
2013. Activities around the research vessel visits can include ship tours for media 
and the public, public programming in the Exploratorium featuring the scientists 
from the vessel, trainings for Exploratorium and NOAA staff, or sample processing 
in the Exploratorium lab. Even when schedules cannot accommodate extra 
activities, the prominence of the ship at the Exploratorium is excellent exposure for 
NOAA. 

• Opportunistic collaborations - The Transit of Venus provided an opportunity for 
the Exploratorium to use a NOAA facility to broadcast the Transit. The event raised 
awareness of the Mauna Loa Observatory. The live webcast and multiple videos 
remain on the Exploratorium website. 

For the Exploratorium, specific benefits as a result of the partnership include: 

• Access to NOAA scientists, ships, equipment and laboratories; 

• Access to observational and modeling data; 

• Incorporating cutting-edge research and real-time data into exhibits on the 
museum floor; and 

• Allowing audiences to experience the results of research as well as the evidence, 
methods and processes scientists undertake to explore the natural world. 

For NOAA, specific benefits as a result of the partnership include: 

• Accelerating fulfillment of NOAA’s mandate under the America COMPETES Act; 

• Engaging with a recognized leader in informal science education in a public 
research and development laboratory; 

• Reaching multiple audiences (e.g., school children, families, young adults, 
educators, policy-makers, web browsers, and life-long learners) via rapid 
prototyping and testing of new ideas; 

• Discovering and applying new models for NOAA to engage the public; 

• Increasing the capacity for NOAA scientists to communicate research, models and 
forecasts to the public; and 

• Providing NOAA greater visibility throughout the museum and science center 
community. 

Lessons Learned 

The cultural differences between institutions were not fully appreciated or understood at 
the start of the partnership, which had the potential to create tension in some situations. 
For example, an exhibit that may be generally unnoticed may be considered by the 
Exploratorium staff as a nice surprise for those who are paying attention to the world 
around them. From a NOAA standpoint, this was not be considered a success or a good use 
of resources. 

A lesson learned quickly was to prioritize ideas and projects, as the breadth of the 
partnership provided many potential areas for collaboration. Initial prioritization included 
ensuring that major NOAA focus areas (e.g., climate, weather, oceans) were covered by 
projects available to a variety of audiences. 
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Current Status of Partnership 

The partnership between NOAA and the Exploratorium has created meaningful 
relationships among several contacts at each institution, ensuring in-kind collaborations 
will likely continue in the future. Benefits to both NOAA and the Exploratorium have 
elevated the status of each partner in a variety of ways: 

• Lasting capacity for both NOAA and the Exploratorium in delivering and 
interpreting ocean and atmospheric science for the public; 

• Capability at the Exploratorium to promote and highlight the work of NOAA in a 
very positive context; 

• The Exploratorium is now capable of contributing original data to ocean and 
atmospheric observing networks and has developed new partnerships with other 
scientific organizations; 

• Contributions to the informal science education (ISE) and science fields; 

• NOAA has an improved reputation in informal science education; 

• The Exploratorium has a good reputation in the science arena; 

• Positive media and public perception of NOAA’s work; 

• Broad engagement in both NOAA and the Exploratorium; and 

• Even if no additional effort were put into the partnership, there would be lasting 
benefits to NOAA, the Exploratorium, and the public. 

Budget constraints present a challenge in continuing travel-based activities including site 
visits, which have been crucial in forming the relationships and direct lines of 
communication between the Exploratorium and NOAA staff and scientists. This may 
impact how the partnership expands to include more components of NOAA in the future, 
and may also result in further exploration into building a robust remote component into 
the partnership. 

Conclusion 

The Exploratorium has established relationships and networks that help provide a 
different level of policy access for NOAA Education officials. The partnership has 
established relationships that provide a different level of access to new or refined 
approaches for audiences, constituents or stakeholders. The partnership is strongly 
supported by connections between people in both organizations committed to sustaining 
the partnership. 
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Case Study IB: NOAA Office of National Marine Sanctuaries (ONMS) and the Oakland 

Museum of California (OMCA) 

A partnership that began in 2005 with an informal conversation broadens NOAA reach to 
diverse populations. 

Background 

The Oakland Museum of California (OMCA) brings together collections of art, history and 
natural science under one roof to tell stories of California and its people. Opened in 1969, 
OMCA is a leading cultural institution of the Bay Area and a resource for the research and 
understanding of California's dynamic cultural and environmental heritage. The museum 
is a community resource for Oakland and East Bay residents, offering community events, 
lectures, docent program, school programs, teacher workshops, and off site explorations in 
California. 

OMCA serves primarily East Bay families (80% of total visitorship), paid admission, 
museum members, and school groups. This museum is becoming more of a tourist 
destination. The annual visitation in 2012 was 150 thousand with a goal to increase it to 
250 thousand. 

• The breakdown of ethnic diversity in the Oakland area is White (31%), 
Black/African American (36%), Asian (15%), Hispanic/Latino (22%), and American 
Indian, Native Hawaiian/Pacific Islander and other races (14%) according to the 
2000 Census. 

• About 40% of participating students are from the Oakland Unified School District 
(OUSD), which includes a diverse student population of over 54 thousand children. 
The ethnic breakdown for OUSD is 47% African American, 29% Latino, 18% 
Asian/Pacific Islander, 6% Anglo/European American, and 1% American 
Indian/Alaskan Native. Nearly 55% of these children come from low-income 
families and qualify for free or reduced price lunches. More than 35% are recent 
immigrants who speak English as a second language. 

The partnership with OMCA evolved from a conversation in 2005 with an interpretation 
specialist, who mentioned that a major renovation was planned in the near future and 
OMCA sought to include more ocean content. This became part of the 5-year exhibit plan, 
which was adopted by the museum’s site management. A proposal of 500 thousand dollars 
was submitted to NOAA Office of National Marine Sanctuaries (ONMS) for Procurement 
and Construction Funds (PAC). Funds were transferred through a Broad Area 
Announcement Grant. In 2007, ONMS awarded funds to the OMCA to build exhibits that 
focus on Cordell Bank in the OCMA Natural Sciences Gallery. 

Through that grant, Jenny Stock of NOAA Cordell Bank National Marine Sanctuary 
(CBNMS) collaborated with OMCA to produce a new permanent exhibit gallery about 
CBNMS, other California sanctuaries, and ocean science and conservation. The goals for the 
exhibits were to: 

1. improve public understanding of the importance of California’s marine and coastal 
resources; 

2. increase awareness of environmental issues along our coastline and stewardship 
opportunities; 
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3. 


provide a venue for students and public to access current marine research 
outcomes; and 

4. implement innovative interpretive approaches to presenting current marine 
science related research and how to become active stewards. 


Results 

NOAA has contributed 500 thousand dollars of funding and 25 to 30 thousand dollars of 
staff time spent on the partnership, funding mechanism, and exhibit development. The 
return on this investment includes exhibit space, ongoing education programming, 
outreach and becoming a hub for learning about National Marine Sanctuaries in 
California. Most importantly, there is a place for people to learn about Cordell Bank, which 
is 20 miles offshore and inaccessible to most people. OCMA provides a built-in audience for 
NOAA to communicate with and influence. 

• OCMA audiences - The partnership broadened the NOAA community by reaching a 
new population in the East Bay. The Natural Sciences Guild, a membership base at 
the museum that attend natural science lectures and field trips, is considering a 
multi-day field trip about Cordell Bank NMS in 2014. 

• Cordell Bank Gallery - The Natural Sciences Gallery reopened in 2013 and includes 
seven places: Oakland, Mount Shasta, Tehachapi Mountains, Coachella Valley, 

Sutter Buttes, Yosemite, and Cordell Bank. The Cordell Bank gallery is a 3 thousand 
square foot gallery of exhibits showcasing the major marine habitats and 
communities located within the Cordell Bank NMS. It also includes exhibits on the 
iconic marine and coastal communities located within all four California National 
Marine Sanctuaries. 

• Education and public proerams - In 2012, OMCA initiated a school program focused 
on Cordell Bank that continues today (as of 2014). This program was funded on 
their own, without additional funding from NOAA. Jenny Stock had a small role in 
consulting about the content of this program. 

Current Status of the Partnership 

This is a currently active partnership (as of 2013). Although all parties would like to 
continue to work together, this is a fragile time right now in the partnership. Now that the 
exhibits in the gallery are open and the original MOA has expired, the partners will 
determine how to work together through programs, community events, and more. There is 
potential for STEM collaborations, teacher professional development, student programs, 
after school programs, public programs and further community engagement. In addition, 
the continued partnership is an opportunity to address following activities: 

• Plan ways to obtain the numbers, evaluation, and impact results. The new draft 
MOA includes such language. Funds were transferred through a Broad Area 
Announcement Grant, but a better arrangement would have been a cooperative 
agreement. 

• Align the partnership with NOAA's current educational output measures, including 
formal education programs, informal education programs, and professional 
development programs. 

Conclusion 

The OMCA has established relationships that helped provide different levels of a) policy 
access to those networks for NOAA Education officials, b) science content knowledge, c) 
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access to new or refined approaches for audiences, constituents or stakeholders in addition 
to helping contribute to NOAA’s workforce development goals. The partnership is strongly 
supported by connections between people in both organizations committed to sustaining 
the partnership, although a lack of funding makes this a fragile time for the partnership. 


29 



Case Study 1C: NOAA and the Science on a Sphere® (SOS) Users Collaborative Network 

A partnership with more than 100 institutions provides effective and efficient 
communication of NOAA-related science and messaging to the public that reaches over 35 
million visitors annually across the globe. A list of the SOS institutions is presented in Table 
2 at the end of this case study. 

Background 

The Science on a Sphere® (SOS) is an animated 68-inch diameter globe designed to show 
dynamic images and data of the atmosphere, ocean, or surface of a planet or moon. 
Current global processes are visualized and displayed on the SOS, making this tool a 
flexible and visually stunning exhibit of real Earth systems science data. Museums, zoos, 
aquariums, and science centers throughout the world are creating exhibits featuring SOS. 

The goal of SOS is to engage audiences with relevant research results and data 
visualizations about Earth to foster a greater appreciation for the interconnectedness of 
the planet, how it is changing, and how people will need to react to and plan for the 
change. NOAA’s partnership with the informal science education institutions displaying 
SOS has helped to advance NOAA’s informal science education objectives. They include the 
following: 

1. partner with informal science education centers to integrate ocean, coastal, Great 
Lakes, weather, and climate science content into their programs through multiple 
platforms and emerging technologies; and 

2. establish and collaborate with networks of informal science education institutions 
to develop effective practices for science content delivery. 

In 2006, OED formed the SOS Users Collaborative Network (SOS Network) to keep the 
groups involved in the SOS education program working together and focused on the 
objectives. The SOS Network consists of institutions that have or will have SOS installed as 
part of a public exhibit, or are currently working with institutions to support the 
application of the technology. Currently, there are more than 100 member institutions in 
this international network. 

OED supports collaboration among SOS Network members through regular workshops and 
meetings and through funding opportunities. The institutions involved in the SOS Network 
discuss the SOS system and software, creation of content, related exhibits, new methods for 
delivering content via SOS, and many other issues relevant to this effort. OED has involved 
the SOS Network in a major initiative to assess the education impact of SOS exhibits. By 
working together, the SOS Network members are advancing their collective ability to teach 
science with SOS. 

The goals of the partnership are to: 

• Build capacity of SOS Network institutions to engage audiences with recent and 
complex Earth system science research results through use of SOS; 

• Provide a mechanism for member institutions to share lessons learned about how 
to use SOS as an effective Earth system science education platform and collaborate 
with one another; 

• Build capacity of docents and presenters to interpret complex data visualizations 
and discuss Earth system science with public audiences; 
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• Provide guidelines for creating easily understood, scientifically accurate content; 
and 

• Evaluate the effectiveness of SOS as a learning tool. 

The SOS Network is largely maintained through the voluntary participation of SOS sites. 
SOS Network members share and collaborate freely with minimal interference from 
NOAA. Services provided by various offices in NOAA and the sharing among Network 
members are largely in-kind. However, OED has funded SOS exhibit development, content 
development, and professional development through its competitive Environmental 
Literacy Grants (ELG) program. ELG also strongly encourages projects to leverage NOAA 
assets like SOS. To date these grants have totaled over 10 million dollars. The professional 
development workshops are funded through the National Marine Sanctuaries Foundation. 

OED manages the SOS Network. Carrie McDougall is the senior program manager and 
leads the effort for OED with assistance from Maria Murray and formerly John 
McLaughlin. Carrie McDougall and John McLaughlin also work on the Environmental 
Literacy Grants program and coordinate funding opportunities with the needs of the 
community. OED works very closely with the SOS Technology Team at NOAA ESRL to 
coordinate activities and participate in bi-weekly team calls. OED also frequently 
exchanges ideas with other NOAA SOS partners at the Pacific Services Center, the 
Environmental Visualization Lab, and Climate Program Office. 

Results 

The Network provides effective and efficient communication of NOAA-related science and 
messaging to the public at science centers and other institutions that reach over 35 million 
visitors annually across the globe - a reach that is only possible through this partnership. 
The partnership is tailored to address NOAA's mission and education goals based on NOAA 
Education Strategic Plan (2009). 

SOS is installed at over 100 institutions worldwide. These institutions include museums, 
science centers, visitor centers, zoos, aquariums, major federal labs, universities, and a few 
K-12 schools. Within the United States, there are 62 SOS sites across 28 states. 
Internationally, there are 42 SOS sites across 15 countries. 

In addition to the institutions that exhibit SOS, the SOS Network includes institutions that 
are government agencies, video production studios, research labs, and informal science 
education specialists. These institutions generally serve the SOS Network to support 
content development for display on SOS. A few groups have been involved in evaluating 
the effectiveness of SOS and SOS presentations for education and engagement. 

Activities of the partnership include: 

• Collaborative meetings - OED has convened the SOS Network for 5 face-to-face 
workshops every 18 months and for 5 smaller meetings annually at the Association 
for Science-Technology Centers conference. These meetings include professional 
development opportunities and other ways to increase the effectiveness of 
education practices using SOS. Between meetings, SOS Network members 
collaborate online by sharing products, resources, evaluations and best practices. 

• Technical support - The SOS Tech Team at NOAA ESRL distributes, installs, and 
provides training for SOS systems year-round. This team is also responsible for 
providing technical support to all SOS facilities. 
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• Grants - OED has provided over $10 million in grants for SOS-related education 
projects. In the early years of the partnership, OED developed focused funding 
opportunities to grow and develop the Network in its nascent stages. Funding has 
been for SOS system installation, exhibit development, content creation, educator 
professional development, public programs, and evaluation. At times, NASA and 
NSF have supported SOS efforts through grants. 

• Dataset and content creation - Collectively, SOS Network members have 
contributed over 400 datasets and content pieces to the dataset catalog. NOAA has 
developed a large portion of these datasets. Network members work together with 
scientists to create stories related to various Earth system science topics. Data are 
visualized in ways that are compelling and comprehendible to the public. 

Evaluation 

The OED and the Institute for Learning Innovation (ILI) conducted a cross-site summative 
evaluation of SOS impacts involving sixteen SOS sites (Haley Goldman, Kessler, & Danter, 
2010). Key results from the evaluation, include: 

• Most participants (71%) reported learning something new and provided specific 
examples as depicted in Figure 17. Learning outcomes did not vary by age, 
ethnicity, or social groups); 

• Those who had seen SOS before were just as likely to report they learned 
something new as those who had not; 

• Most respondents (82%)stated that the spherical display changed their 
understanding of the information; 

• SOS supports understanding complex processes and phenomena; and 

• Visitors who saw a facilitated presentation were significantly more likely to report 
they learned something new (87%) than those who viewed an unfacilitated 
program (66%). 

Figure 17. Categories of Specific Things Visitors Learned. SOS Summative Evaluation. 
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Return on Partnership 

NOAA’s role: 


• The SOS Tech Team provides SOS software updates, SOS equipment, installation, 
and technical support; 

• NOAA scientists and educators provide content knowledge for exhibit and content 
development; 

• NOAA produces data and data visualizations; 

• NOAA catalogs all content (data visualizations, scripts, etc.); 

• NOAA staff plans and executes Network workshops/meetings; 

• NOAA staff facilitates collaboration and communication across Network; and 

• NOAA offers funding opportunities open to the SOS Network. 

The SOS Network’s role: 

• Network members directly interact and share assets, science, and information 
related to NOAA work to more effectively and efficiently communicate with public; 

• Network members contribute to NOAA dataset catalog; 

• Network members conduct evaluations on the effectiveness of SOS (a NOAA 
product) as an educational tool; 

• Network members act as trusted providers of scientific information and provide 
audience-appropriate data presentation and context; and 

• Network members provide exhibit space and credit signage for NOAA’s Science On 
a Sphere®. 

Both NOAA and Network members produce and share video content and supporting 
materials (e.g., scripts, activities). 

Lessons Learned 

Voluntary participation in a networked partnership is critical. While some partners take 
leadership roles, not all SOS institutions are active partners. Institutions and individuals 
can broker their own involvement based on their priorities and availability. Less active 
partners still can gain through their loose affiliation by using contributed content and tips. 

A broad network of institutions that are disparate geographically and in their individual 
missions can be highly collaborative if their common goal is a very focused one. While the 
specific functioning and needs of each institution are different, the convener of the 
network (OED) needs to be able to synthesize these needs and hear the signal above the 
noise to determine common goals and initiatives. 

Regular, though infrequent, face-to-face meetings have been an important activity for 
those who are able to participate. Participants report valuing the forum for networking, 
collaborating, sharing ideas, and refocusing their thoughts and plans on the central topic of 
the partnership (SOS). 

With so many institutions in a networked partnership, it has been impossible to tie too 
much of the partnership to specific individuals. As time goes on, the institution’s 
commitment to the partnership may remain even though personnel at these institutions 
change. Flexibility is needed to engage new people. 
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In order to keep advancing the work of the partnership, it may be beneficial to engage 
complementary disciplines and expertise outside the network. Some examples of experts 
that have helped the SOS Network include: artists, interpreters, evaluators, movie 
producers, scientists, social scientists, policy makers, and industry. 

Current Status of the Partnership 

This partnership is active and fully operational. To maintain the SOS Network and plan 
regular Network workshops, the partnership requires effort from people at all of above 
mentioned NOAA offices, which totals to over 1 FTE. 

The SOS Network was developed to build capacity of informal science education centers to 
utilize Earth system science data effectively in their education efforts. The Network 
continues to grow through new SOS installations and an expanding list of organizations 
supporting SOS work. Both the hardware and software technology is continually updated 
in response to the needs of the SOS Network members. The SOS Network likely will 
continue to operate as long as SOS is a supported as an educational tool by the SOS Tech 
Team and the museum community. 

While OED has made significant investments in the Network, the financial contributions 
that are made help sustain the face-to-face focused collaborative meetings that Network 
members continue to value. The voluntary and collaborative nature of the Network does 
not necessitate financial contributions. If OED’s involvement were to cease tomorrow, SOS 
Network members would continue to collaborate. The in-kind management work that OED 
provides does help the Network address cross-institutional needs and priorities in a more 
unified and synergistic way. 

Conclusion 

SOS is a high-return NOAA partnership with institutions (as opposed to individuals), 
requiring different methods of fostering relationships. 
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Table 2: SOS Network Sites. 


Governor Tauese P.F. Sunia Ocean Center 
Instituto Oceanografico da Universidade de Sao 
Paulo 

Museum of Natural History 
TELUS World of Science 
Techmania Science Center 
Visual Climate Center 
Heureka, The Finnish Science Centre 
Cite de L'Espace 

Cite des Sciences et de L'lndustrie 
Museo delle Scienze 
Discovery Center 
Climate Institute - Puebla, Mexico 
Climate Institute - Cuernavaca, Mexico 
Climate Institute - Chilpancingo, Mexico 
Climate Institute - Atlacomulco, Mexico 
Climate Institute - Metepec, Mexico 
Climate Institute - Casa de la Tierra, Veracruz, 
Mexico 

Climate Institute - Planetario de Morelia, Morelia, 
Mexico 

Climate Institute - Acapulco, Mexico 
Climate Institute - Texcoco, Mexico 
Climate Institute - Valle de Bravo, Mexico 
Climate Institute - Villahermosa, Mexico 
Climate Institute - Oaxaca, Mexico 
Climate Institute - Ciudad Victoria, Mexico 
Climate Institute at the Natural History Museum 
Beijing Huaxinchuanzi Technology Co., Ltd. 

China Beijiko Meteorological Museum 
China Maritime Museum 
China Science and Technology Museum 
China Weather Channel, Huafeng Group 
Dongguan Meteorology and Astronomy Museum 
Fuzhou Science and Technology Museum 
Daegu National Science Museum 
Gwacheon National Science Museum 
Gwangju National Science Museum 
KIGAM Geological Museum 
National Youth Space Center 
Science Centre 
Our Planet Centre 
Central Weather Bureau 
National Museum of Marine Science and 
Technology 

National Museum of Natural Science 

Science Museum 

Alaska State Museum 

Aldo Leopold Nature Center 

Aquarium of the Pacific 

Bay Education Center 

Bishop Museum 

Boonshoft Museum of Discovery 
Clark Planetarium 
Climate Corporation 


Cyberinfrastructure Building at Indiana University 
Danville Science Center 
Denver Museum of Nature & Science 
Detroit Zoological Society 
Discovery Science Center 
E.O. Wilson Biophilia Center 
Fiske Planetarium and Science Center of Colorado 
University 

Galaxy E3 Elementary 
Grand Canyon Visitor Center 
Great Lakes Maritime Heritage Center 
Great Valley Museum 
Harsco Science Center 

Houston Museum of Natural Science at Sugarland 
Imagination Station Science Museum 
Imiloa, Astronomy Center of Hawaii 
Infinity Science Center 
International Museum of Art & Science 
James Madison University 
Lawrence Hall of Science 
Maryland Science Center 
McWane Science Center 
Microsoft Visitor Center 
Museum of Science and Industry 
NASA Ames Exploration Center 
NASA Goddard Space Flight Center Visitor Center 
NASA Wallops Flight Facility Visitor Center 
Nauticus - The National Maritime Center 
NOAA Earth System Research Laboratory 
NOAA Headquarters 
NOAA Inouye Regional Center 
NOAA National Severe Storms Laboratory 
NOAA Ted Stevens Marine Research Institute 
(NMFS) 

North Carolina Aquarium 
Nurture Nature Center 
Ocean Explorium 

Oregon Museum of Science and Industry 

Orlando Science Center 

Pacific Science Center 

Point Reyes National Seashore 

Science Central 

Science City at Union Station 

Science Museum of Minnesota 

Science Museum of Virginia 

Smithsonian National Museum of Natural History 

Smithsonian's National Zoo 

South Florida Science Center and Aquarium 

Space Foundation Visitors Center 

St. Charles High School 

St. Paul's School 

Tech Museum of Innovation 

US Astronaut Hall of Fame 

Wild Center 

Wildlife Experience 
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Case Study 2A: NOAA and the American Meteorological Society (AMS) 


A partnership between NOAA and the American Meteorological Society (AMS) develops 
and implements courses for pre- and in-service K-12 teachers with strong content in 
atmospheric and ocean sciences. It has reached an estimated 140 thousand teachers and 
4.9 million students directly or indirectly since the inception of the program in 1990. 

Background 

The AMS is a non-profit professional and scientific organization representing those in 
atmospheric, oceanic, hydrologic, and related sciences. Like most professional 
organizations, AMS publishes peer-reviewed publications, holds various meetings and 
conferences for its members and other professionals involved in scientific research, 
forecasting and related work. AMS offers numerous educational programs for K-12 
audiences, focusing primarily on in-service teachers, and postsecondary audiences. It 
creates introductory college-level weather, ocean, and climate courses offered by higher 
education institutions, with a special focus on smaller liberal arts colleges, community 
colleges, and minority-serving institutions. All educational initiatives include a component 
directed toward increasing participation of underrepresented minorities in science and 
science teaching. 

The AMS Education Program was established in 1990. This same year marked the 
beginning of Project Atmosphere and the close working partnership between AMS and 
NOAA. With a NSF grant and the in-kind support of NOAA, a two-week summer precollege 
teacher workshop on weather topics was conducted at the NWS Training Center (NWSTC) 
in Kansas City. The NWSTC summer workshop had already been in existence since 1984, 
run by the State University of New York [SUNY] Brockport with NWS in-kind support. It 
became the first major initiative of Project Atmosphere. In 1994, again with primary 
support via grant from NSF and in-kind support from the U.S. Navy and NOAA, the Maury 
Project: Exploring the Physical Foundations of Oceanography Program was started at the 
U.S. Naval Academy. These two summer programs have been held annually since then and 
provided the initial training of master teachers who were instrumental in the 
implementation and continued offering of the AMS suite of DataStreme courses. 

With the reduction over time of support from NSF, NOAA began providing direct funding 
in support of both summer training programs. This funding came from different line 
offices, beginning with annual ad hoc grants from NESDIS, NWS, NMFS, NOS, and OAR. In 
2001, the AMS/NOAA Cooperative Program for Earth System Education was proposed in 
order to draw together the various NOAA and AMS Education Program cooperative 
components. The working relationship between AMS and the NOAA Office of Education 
(OEd) was formalized with a Memorandum of Agreement (MOA) on 13 September 2004 
and signed by John J. Kelly, Jr., Deputy Under Secretary, NOAA/DOC. The MOA states that 

"... the Parties will support a cooperative program of projects and activities in 
ocean, coastal, atmospheric, hydrologic, and Earth system science education, aimed 
at preparing citizens in our society to understand and act on information related to 
our planet’s dynamic air and water systems and how they affect every aspect of 
our lives.” The MOA formalized the de facto working relationship that has existed 
between NOAA and AMS for many years. 
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The DataStreme programs have been developed and continue to be conducted with 
technical assistance and various kinds of support from all NOAA line offices and hundreds 
of NOAA personnel. The first DataStreme course, DataStreme Atmosphere, was 
implemented in 1996 via Local Implementation Team (LIT) leaders, who were trained at 
Project Atmosphere summer workshops. NOAA provided in-kind support for DataStreme 
Atmosphere, particularly in the annual summer training of LIT leaders. Each LIT is 
composed of at least one master precollege teacher and most include a professional 
meteorologist, hydrologist, oceanographer or environmental scientist. NWS field offices 
have hosted many of these meetings. In 2002, NOAA began providing direct financial 
support for this partnership via a grant. In 2004, the National Weather Service’s National 
Center for Environmental Prediction became the primary provider of over 175 real-time 
meteorological products delivered via the course webpage, including customized maps, 
charts, imagery, and text. 

This partnership has evolved from in-kind support of teacher professional development 
programs through the direct involvement of NOAA personnel and facilities to financial 
support in the form of grants and cooperative agreements. NOAA’s early awards to AMS 
were non-competitive, but since 2005 OED has provided a competitive award vehicle for 
one-NOAA financial support. The DataStreme Ocean course started in 2005 with with a 
3-year award from NOAA’s Environmental Literacy Grant Program. Development and 
subsequent implementation of DataStreme Ocean and Atmosphere courses has been 
almost entirely supported by NOAA both through in-kind and annual financial support. In 
2007 an institutional award to AMS was established competitively and then renewed in 
2012 for another five years. The evaluation criteria of the funding announcement that 
established the initial institutional award stressed the importance of the involvement of 
NOAA personnel and other NOAA assets as well as the use of partnerships to leverage 
NOAA’s investment. Since 2007, six NOAA offices (NESDIS, NWS, NMFS, NOS, OAR and 
OED) have contributed equally to program support, with award administration through 
OED. 

Results 

Conservatively over 14 thousand educators, primarily in K-12, have completed the 
DataStreme Atmosphere and Ocean courses. Teachers have been asked to report on the 
number of other teachers they train on the course materials and the number of students 
reached by themselves or these secondary recipients. To date, an estimated 140 thousand 
teachers and 4.9 million students have benefited directly or indirectly from these courses. 

Since 2002, NOAA has provided almost $7 million in direct financial support to AMS to 
support its K-12 teacher professional development and leadership training programs. 

The in-kind contribution of 275 NOAA professionals involved in supporting DataStreme 
Atmosphere and DataStreme Ocean courses is estimated to range from 96 to 120 hours per 
year per person for an average of 5 years since 1996. The time estimate is based on the 
average volunteer time for one NOAA LIT member assuming a contribution of 4 to 5 hours 
per week in each 12-week semester. NOAA personnel have also provided time and 
expertise in the review of AMS’s course materials, but this contribution has not been 
quantified. Nevertheless, assuming $150K for the annual cost per federal FTE (a standard 
estimate), NOAA’s minimum in-kind contribution has been $9.52M - 11.90M since 1996. 
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The in-kind contribution is likely much greater because the estimate does not include the 
review of course materials and support provided at NOAA locations for LIT meetings, as 
well as the summer institutes that train the master educators who in turn become LIT 
members. Additionally, this figure does not include the in-kind cost of NCEP’s production 
of real-time data visualizations and explanatory text. 

A conservative per-teacher cost estimate ranges from $1,158 to $1,325. NOAA’s real and 
in-kind investment has enabled the integration of NOAA data, data products, and 
educational resources into these teachers’ classrooms and beyond, thus introducing 4.9 
million students in the US to NOAA-related sciences and NOAA products. 

Further, NOAA’s investment in these courses has been matched almost dollar-for-dollar by 
AMS. Through SUNY-Brockport, AMS has been able to offer tuition waivers for their course 
credits. This offer of three free course credits is a huge incentive for educators to complete 
the semester-long courses. AMS’s cumulative in-kind contribution in tuition waivers for 
Datastreme Atmosphere and Ocean course participants is $20.2 million, with an an average 
participant cost of $1,373. 

Lessons Learned 

One of the reasons for AMS’s continued success in sustaining this partnership over such a 
long time is their continual cultivation of supporters among NOAA leadership as well as 
those individuals providing in-kind support “on the ground.” The deep and long-lasting 
connections with individuals at NOAA have enabled AMS to sustain some financial support 
even while overall funding has gone down. AMS continues to focus on long-term 
involvement of the teachers who have taken their courses and local implementation team 
members who have helped them implement their programs. Additionally, AMS has 
leveraged the funding provided by various federal agencies across its suite of programs 
that they offer in support of K-12 and higher education. This has been a strong selling point 
with NOAA leadership. 

Current Status of the Partnership 

AMS has been a driving force behind this partnership and AMS educational staff members 
have actively cultivated numerous connections throughout the agency. Consequently, the 
biggest challenge in recent years has been sustaining a budget from across all line offices 
during the strained financial situation facing the agency’s budget, especially for its 
education programs. An additional challenge from OED’s perspective has been managing 
the process for obtaining funds from across the line offices while continuing to encourage 
AMS to conduct a more robust evaluation of their courses’ impacts despite recent budget 
cuts. 

NOAA personnel and NOAA data products, educational resources and other assets will 
continue to be a key piece of the DataStreme courses. The question is whether AMS will be 
able to shift the burden of financial support to other sources outside of NOAA. The 
institutional award established in 2007 was intended to show NOAA’s long-term 
commitment to the program with the expectation that over time the dependence on NOAA 
funding to sustain the routine operations of the Datastreme courses would decline. AMS is 
currently developing revenue streams from the sale of their course materials, as textbooks 
and more recently e-books, from multiple programs to sustain these operations. 
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Conclusion 

The NOAA partnership with the American Meteorological Society (AMS) meets the criteria 
of a high-return partnership, in that the benefits to NOAA and partners outweigh the 
investment and involvement for all parties involved. The partnership provides benefits 
national, regional, local initiatives as described in this case study. Partnership success has 
been contingent on connections among people in both agencies. 


39 



Case Study 2B: National Weather Service (NWS) and PLANIT NOW 


A partnership between the National Weather Service (NWS) and PLANIT NOW with the 
NOAA Office of Education, National Weather Service Outreach Program, and the American 
Meteorological Society-Education Office to produce the Young Meteorologist Program. 

Background 

PLANIT NOW (PIN) is an international disaster preparedness nonprofit organization whose 
public awareness campaigns help individuals and communities in the United States and 
worldwide understand the best strategies for surviving natural and human-made hazards. 
PIN’s hallmark public education initiative, the Young Meteorologist Program is a free 
online computer game that educates and empowers children in grades K-8 about 
severe-weather science, weather confidence and safety. 

Founded in 2005 after Hurricane Ivan devastated Grenada, PIN assists communities at risk 
of being impacted by disasters including hurricanes, flooding, lightning, tornadoes, winter 
storms, earthquakes, fires and other natural and man-made disasters. It provides relief in 
the form of scholarships to meteorology students and students affected by severe weather 
events. PIN has advocated empowering people for severe weather events with 
eye-catching, engaging information. The program portfolio includes public awareness 
campaigns that help individuals and communities understand the best strategies for 
surviving natural and human-made hazards. PIN works with public, government, 
nonprofit and private partners from fields as diverse as disaster relief, insurance, social 
work, disaster studies and meteorology to disseminate important information to help those 
at risk of disasters prepare their families and communities. 

The NWS was introduced to the PLANIT NOW (PIN) organization by the AMS. After 
requesting NOAA General Counsel guidance, it was determined both organizations (PIN 
and NWS) had similar education goals and both organizations would benefit from a 
partnership. An MOU was signed by both organizations and NWS appointed the Outreach 
Program Manager as the science advisor to PIN. 

Unique for the National Weather Service, it has granted non-exclusive, royalty-free, 
worldwide rights to PIN to use all versions of its Owlie Skywarn trademark and severe 
weather slogans, “Turn Around Don’t Drown.” to produce the Young Meteorologist 
Program (YMP) and in all promotional activities related to the YMP. YMP is a digital 
evolution of the National Weather Service’s famed Owlie Skywarn publication, which has 
educated children about the dangers of severe weather since the 1970s. A goal of the 
partnership is to associate Owlie with severe weather preparedness in the same way the 
US Forest Service and their Smokey Bear is associated with forest fire prevention. The YMP 
team consists of science teachers and educators, meteorologists, software developers, 
marketing professionals and celebrities from the entertainment industry. 

YMP takes children in third through eighth grades on a “Severe Weather Preparedness 
Adventure” where they encounter lightning, hurricanes, tornadoes, floods, and winter 
storms. The YMP centerpiece is an interactive online game featuring a 21st-century Owlie 
Skywarn, a quirky owl and junior data collector for the weather center who helps kids 
prepare for real life severe weather and natural hazards. In addition, through the program 
website, young meteorologists will be presented with opportunities to put their new 
knowledge to work through hands-on activities and community service projects. 
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YMP offers a safe, non-threatening learning environment to show children how to prepare 
to protect themselves, help their families, and their communities when a real storm or 
other forms of severe weather comes to their city or town. The YMP stands out from other 
preparedness programs because it is aimed at children as the gateway into families to take 
action, i.e. creating a family disaster supply kit. Resources that educators, parents, and 
meteorologists can use to guide their newly appointed Young Meteorologists are available. 

Results 

NWS’s long term goal is to significantly reduce the NWS Owlie publication costs and thus 
save the federal government annual printing costs. Annual printing cost savings to the 
federal government range from 15 thousand to 20 thousand dollars annually. Since Owlie’s 
inception in 1974, NWS estimates it has printed over 1.5 million copies of the Owlie 
publication, 30 to 40 thousand copies annually. NWS expects the YMP to be even more 
popular since it will be available for free on the Internet. 

YMP went on lin e September 2012. PIN has installed Google Analytics to collect data on 
visits to the YMP website. For example, preliminary metrics collected during the period of 
July 2011 to February 2013 there were a little less than 11 thousand visits to the YMP web 
site. NWS and PIN are working on methods to provide routine metrics. This is still a work 
in progress. 

Initial testing, including classrooms, has shown YMP is popular with all family members 
including children, parents and grandparents for a “family” bonding session. Players who 
complete the game earn a Young Meteorologist Certificate. 

Data was collected from the NWS field office staff using YMP in local community outreach 
events, such as school visits, and from AMS Atmospheric Education Resource Agents who 
used the YMP material in their classrooms. NWS and PIN continue to promote the YMP 
activity at conferences and celebrity events. NWS takes the lead in promoting YMP in 
science and education conferences. 

PIN has provided the following: 

• YMP game software; 

• YMP high quality graphics for YMP banners and exhibits; 

• YMP bookmarks for NOAA/NWS education and outreach events; and 

• PIN Owlie costume, available to NOAA/NWS for outreach events nationwide. 

NOAA has contributed approximately 1 FTE in staff time. NOAA has also provided 
materials (e.g., curriculum, activity kits, literacy guides, outreach materials), expertise (e.g., 
scientific, educational), datasets and data visualizations. The PIN/YMP activity is integrated 
into NOAA and NWS exhibits at conferences and workshops. NOAA/NWS Scientific content 
(e.g., publications-imagery) are provided and integrated into the PIN/YMP web site, which 
is accessed by teachers, students, educators and the public. 

NWS has provided the following: 

• Accurate and current science: hurricanes, tornadoes, winter storms, lightning, 
floods; 

• Safety and preparedness content from Red Cross and FEMA; 
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• Marketing and promotional opportunities within the “Weather Enterprise” 
community; 

• New P!N Team members (National Weather Association-Midland Radio); 

• Promotion of the YMP on the NWS Weather Forecast Offices home page; and 

• YMP Facebook and Twitter presence. 

PIN's professional experience in marketing and promotion allows NOAA and NWS 
messages to reach large, broad audiences. High visibility media entertainers such as Kenny 
Chesney, Michael Douglas and Morgan Freeman and PIN celebrity boosters provide 
opportunities for NOAA and NWS to promote severe weather safety to markets and 
organizations not usually linked with government agencies. For example, the 2012 
Celebrity Golf tournament had NWS weather safety messages (e.g., “When Thunder Roars, 
Go Indoors”) posted at every hole during the tournament. 

PIN is constantly seeking events outside the "weather enterprise community" to expand the 
reach of NOAA Education and NWS Outreach. PIN promotes the NWS Weather Ready 
Nation Campaign and its messages at venues outside of the "weather enterprise" such as 
the Federal Government's Public Service Recognition Week-DC. 

Lessons Learned 

Frequent, honest communication between all members of the partnership (including 
scientists and administrators) is critical. Face-to-face meetings are critical. Modern 
technologies (e.g., emails, webinars) are not a substitutes for face-to-face meetings. 

The composition of the team is important, each having open minds and being respectful of 
different opinions. Strong, dynamic team leadership which can reach consensus is a must. 

Making sure the partnership stays relevant throughout the years is important. During the 
course of the partnership, budgets change, new capabilities emerge, and surprises pop up 
along the way. The team must be nimble and respectful of other team members. There 
could also be an element of growth into new unexpected areas which the partnership did 
not originally anticipate. 

Current Status of the Partnership 

The partnership began in 2010, and is active and fully operational. Strategic planning 
continues between both organizations on how the promote the NWS Weather Ready 
Campaign nationwide using high visibility venues such as music concerts, sporting events 
and TV talk shows. This is still a work in progress, looking to involve more organizations 
within the weather enterprise, such as Midland Radio Corporation (NOAA Weather Radio 
manufacturer) and The Weather Channel. 

Conclusion 

The partnership between NWS and PIN allows the NWS to reach out to communities, 
organizations outside of the weather enterprise community. This is consistent with the 
recent National Academy of Public Administration's report: "Assuring the Capacity of the 
NWS". The report for the NWS "...considerable engagement and close collaboration with 
public and private sector partners. The NAPA report further states the NWS will need the 
support and cooperation of its partners to achieve its vision of a "Weather Ready Nation." 
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Case Study 3A: EPA Great Lakes National Program Office (GLNPO) and the Center for 

Great Lakes Literacy (CGLL) 

A partnership between the EPA Great Lakes National Program Office (EPA GLNPO) and the 
Center for Great Lakes Literacy (CGLL) ensures continued support for Great Lakes literacy 
collaborative efforts. 

Background 

The Great Lakes Literacy effort had its origins in the Ocean Literacy movement, where 
hundreds of scientists and educators contributed time and expertise to develop a concise 
framework for conveying the most important ocean science principles and interconnected 
concepts that all citizens should know. Educators in Centers for Ocean Sciences Education 
Excellence (COSEE) Great Lakes gratefully acknowledge the groundbreaking work of those 
who contributed to Ocean Literacy. 

While the Ocean Literacy principles and concepts provided useful guidance for teaching 
about the marine environment, educators using ocean literacy principles in the Great 
Lakes found themselves stretching to say “ocean” when teaching on a lake. After aquatic 
educators in Ohio identified a need for a place-based environmental literacy framework 
for Lake Erie, the Lake Erie Literacy Principles (LELP), based on the Ocean Literacy 
framework, were developed to guide a unified strategic plan for Lake Erie education and 
outreach. Concurrent with the development of the LELP, COSEE California, a guiding force 
in the Ocean Literacy movement, recognized the need to expand the utility of the Ocean 
Literacy Principles for the Great Lakes region. With the support of COSEE California, 
COSEE Great Lakes education leaders in the Great Lakes Sea Grant Network examined the 
Ocean Literacy Principles and LELP and drafted a baseline set of principles for Great Lakes 
literacy in October 2009. The draft was reviewed and edited by more than 80 scientists and 
educators throughout the eight Great Lakes states. Ohio Sea Grant educators synthesized 
and organized feedback, ultimately compiling Great Lakes Literacy Principles and 
Fundamental Concepts for Great Lakes Learning. 

For the development of Great Lakes Literacy Principles, the concepts of Ocean Literacy 
were followed as closely as possible, acknowledging the leadership role of the Ocean 
Literacy community and wider public input to the project. Additionally, the Earth Systems 
background of Ohio educators, beginning with Lake Erie principles, contributed to a need 
to say more about connections of humans with the lakes than was encompassed in Great 
Lakes Literacy Principle #6: The Great Lakes and humans in their watersheds are 
inextricably interconnected. From the Lake Erie document, then, Great Lakes Literacy 
adopted Great Lakes Literacy Principle #8: The Great Lakes are socially, economically and 
environmentally significant to the region, the nation and the planet. This addition allows 
for greater inclusion of the environmental history of the lakes and their role in the 
development of the region’s history, economy, and regional identity. 

With the elimination of NSF funding for COSEE, COSEE Great Lakes’ successful programs 
and resources were reinvigorated in 2012 as the new Center for Great Lakes Literacy 
(CGLL). NOAA Sea Grant educators from across the Great Lakes basin (New York, 
Pennsylvania, Ohio, Michigan, Illinois, Indiana, Wisconsin, and Minnesota) who were 
previously part of COSEE Great Lakes, cultivated a rich collaborative relationship with 
EPA's Great Lakes National Program Office (GLNPO) in Chicago. CGLL strives to develop a 
Great Lakes-literate public capable of effectively contributing to the environmental, 
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economic and social sustainability of the Great Lakes. The group fosters informed and 
responsible decisions that advance basin-wide stewardship by providing hands-on 
experiences, educational resources and networking opportunities. These activities promote 
Great Lakes literacy among an engaged community of educators, scientists and citizens. 
The center is primarily funded by the Great Lakes Restoration Initiative (GLRI) money 
administered via EPA GLNPO. 

Results 

EPA GLNPO provides access to the R/V Lake Guardian for five years. Focusing on a 
different lake each year, fifteen teachers spend one week aboard the vessel participating in 
the annual Great Lakes Shipboard Science program. The contribution of technical logistics, 
staff time and scientific expertise is essential in maintaining this highly successful and 
unique professional development opportunity. EPA GLNPO also partners with Sea Grant on 
the Limno-Loan program, where past participants in CGLL events can request use of one of 
four HydroLab multiparameter sondes for assessing water quality for use in their own 
educational setting. While Sea Grant coordinates the overall program, EPA GLNPO has 
purchased and maintains the equipment, as well as provides technical support. 

NOAA’s Great Lakes Sea Grant educators provide support for workshops, including 
participant selection, trip agenda, NOAA curricula, NOAA scientists to work with teachers, 
land-based facilities for professional development, knowledge and expertise of Great Lakes 
environmental issues, and educational resources to share with participants. NOAA Sea 
Grant educators also conduct “Teachable Moments,” which are one to two day workshops, 
and sponsor opportunities for stewardship, citizen science and outreach events (e.g., Great 
Lakes Awareness Day). Staff time, website development, communications and publication 
development is also provided as in-kind support. Specifically, Ohio Sea Grant conducts a 
week-long course on Great Lakes Education, held at Stone Laboratory on Lake Erie, and 
provided facilities, equipment, staff, materials and relevant curricula. Ohio Sea Grant 
provided 2 thousand dollars for instructor salary for a Great Lakes Education course, while 
GLRI funding from EPA provided 5 thousand dollars in participant support. NOAA Sea 
Grant staff time in maintaining the CGLL partnership is approximately less than 0.2 FTE 
across the basin and is often combined with other program efforts. 

Highlighted effects of this support includes: 

• Allowing for the continuation of the Great Lakes Shipboard Science workshop for 
educators aboard the R/V Lake Guardian; 

• Enhancing place-based learning opportunities (including shipboard science and 
citizen science programs, as well as environmental stewardship events), in eight 
states throughout the Great Lakes region; 

• Funding Great Lakes Awareness Days (GLADs) and Teachable Moment workshops 
in regional states; 

• Supplying technical equipment for place-based learning opportunities about water 
quality; 

• Providing classroom teachers the opportunity to attend and present at the 
International Association for Great Lakes Research (IAGLR) Conference; 

• Providing travel and tuition support for participants in The Ohio State University’s 
(OSU) Great Lakes Education Workshop; and 
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• Aiding in the revitalization of the COSEE Great Lakes website as it transitions into 
cgll.org (a hub for regional NOAA/Sea Grant professional development 
opportunities). 

Lessons Learned 

Coordinating Great Lakes educational efforts among 5 lakes, 7 Sea Grant offices and eight 
states, along more than 10 thousand miles of shoreline and throughout 247 thousand 
square kilometers of watershed, and for more than a quarter of the U.S. population takes 
teamwork. CGLL is a true team of educators and the concept of captains and crew is 
unknown to the group. 

Lesson 1: Establish a few key programs that will be standard throughout the group, but allow 
partners to branch out with their own specialties. 

The EPA GLNPO-CGLL partnership is built upon the successful events and best 
management practices that originated with COSEE Great Lakes, as well as the individual 
strengths of the seven Sea Grant state programs in the Great Lakes network. This 
collaborative model maintains the key programs for which COSEE Great Lakes was known 
for (e.g., Shipboard Science on the R/V Lake Guardian, Teachable Moments, land-based 
watershed workshops, and IAGLR’s Educator Day), while allowing individual states to 
broaden their education and outreach efforts in their areas of expertise. For instance, one 
state may have the equipment and facilities to host professional development for pre- and 
inservice educators, while another state may focus on citizen science or stewardship 
projects for K-12 students, and a third state may be prefer to enhance Great Lakes literacy 
via webinars and social media. All members of the partnership share a vision to develop a 
Great Lakes-literate public capable of effectively contributing to the environmental, 
economic and social sustainability of the Great Lakes. However, supporting each state 
program’s unique strategies and established successful projects to achieve this goal allows 
CGLL - and thus NOAA - to expand its community of practice to a larger network of Great 
Lakes educators, scientists, and citizen science groups who will become committed to 
stewardship of our greatest freshwater resource. 

Lesson 2: Emphasize evaluation. 

Frequent reporting requirements are present at all levels (state, regional, national). 
Formative and summative evaluation not only provides constructive feedback for 
programs, but also serves as a framework for needs assessments or justification for future 
funding opportunities. Clearly identifying areas of need or gaps in aquatic education 
initiatives allows CGLL to develop competitive proposals for additional financial support 
and broaden their scope of impact. 

Logic models assist in clearly identifying measurable outcomes for projects. Furthermore, 
CGLL is developing a system of common metrics to ensure objectives are achieved, 
including core questions for use in all CGLL programming and a bank of elective questions 
specifically designed for different types of activities (e.g., stewardship projects, teacher 
professional development workshops, or ) and various audiences. Investment in online 
survey software to streamline data analysis and house a question bank saves time and 
allows members of the partnership to efficiently share vetted survey questions and 
formats. 

Moreover, CGLL focuses on following up with participants for an extended period after 
programming to collect data on the achievement of long-term outcomes. Ohio Sea Grant is 
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facilitating an extended longitudinal survey examining the impacts of the Great Lakes 
Shipboard Science workshop on educators and their students since its inception in 2006. 
These strategies ensure a richer understanding of the effects of NOAA’s regional efforts to 
increase Great Lakes literacy. 

Lesson 3: Communicate regularly 

An initial strategic planning meeting hosted by EPA GLNPO set the course for regular and 
consistent communication among the group. Although CGLL spans a large geographic area, 
members use their attendance at regional and national events (e.g., Great Lakes 
Place-based Education Conference, Great Lakes Sea Grant Network Meeting, or National 
Marine Educators Association Annual Conference) as opportunities to convene, reflect on 
CGLL events and plan future endeavors. 

CGLL also recognizes the strengths of individual Sea Grant programs and supports those 
programs in completing specific tasks that sustain the group as a whole. Illinois-Indiana 
Sea Grant recently developed a poster presentation introducing CGLL and made it 
available for all members to use. Frequent conversations (via phone or email) between 
partners update everyone on progress and happenings, as well as encourage accountability 
and continued completion of tasks. 

Conclusion 

The partnership between the EPA Great Lakes National Program Office and the Center for 
Great Lakes Literacy catalyzes established relationships that integrate Great Lakes science 
and technical expertise with effective educational pedagogy. EPA GLNPO administers the 
GLRI funding that supports the CGLL collaborative effort. It helps NOAA provide a more 
expansive level of access to new or refined approaches for audiences, constituents and 
stakeholders to protect, maintain and restore Great Lakes ecosystems. 

The partnership is strongly supported by connections between people in both 
organizations committed to sustaining the collaborative efforts. Furthermore, it allows 
Great Lakes Sea Grant Educators to achieve all six outcomes under NOAA’s educational 
goal of advancing environmental literacy. 

The development of a locally sustained partnership following the conclusion of grant 
funding is continued and sustained by rich, collaborative relationships among several 
regional groups focused on the same goals. 
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Case Study 3B: Channel Islands National Marine Sanctuary (CINMS) and Channel 

Islands National Park (CINP) 

A partnership between Channel Islands National Marine Sanctuary and Channel Islands 
National Park created a cooperative educational program to the benefit of all partners 
involved. 

Background 

Channel Islands National Park (CINP) is a special terrestrial and marine protected area of 
national and global significance, administered by the National Park Service. The park 
consists of 250 thousand acres of land and ocean environment encompassing the Anacapa, 
Santa Cruz, Santa Rosa and San Miguel Islands. CINP monitors and protects threatened and 
endangered species, restores ecosystems and preserves those natural and cultural 
resources for current and future generations. 300 thousand visitors to park visitor center, 
30 thousand people to the islands and an additional 60 thousand to park and sanctuary 
waters through various commercial and recreational boating activities. 

In 2000, Channel Islands National Marine Sanctuary (CINMS) established a MOU with 
whale watch and marine excursion vessel operators who operate out of Santa Barbara 
Harbor, Ventura Harbor, and Channel Islands Harbor. The MOU created a cooperative 
educational program between CINMS and the vessel operators. CINMS trains and 
schedules specially trained volunteers to represent the sanctuary and educate passengers 
on board vessel operators, at local outreach events, and to serve as citizen scientists 
collecting opportunistic marine mammal sighting data. 

In 2002, an MOU between the CINMS and CINP was established to merge the programs to 
include joint training, funding and scheduling of volunteers. The program name was 
changed to the Channel Islands Naturalist Corps. Volunteers represent both agencies on 
board CINP concessionaire vessels as well as non-concessionaire marine excursion vessels 
operating in the Santa Barbara Channel and CINMS and CINP waters. The partnership 
enables CINMS and CINP to have more efficient use of community volunteers and allows 
each agency to expand its educational services while also providing additional 
opportunities. 

Channel Islands Naturalist Corps (CINC) are a group of specially trained volunteer ocean 
stewards dedicated to educating passengers on board vessels visiting the Channel Islands 
National Marine Sanctuary and National Park. Members provide education about the 
unique marine life found in sanctuary and Park waters to thousands of local residents, 
tourists, and school children annually. CINC volunteers also participate in numerous local 
outreach events and collect valuable research on marine mammals and other important 
sanctuary and Park resources. Volunteers accepted into the program are specially trained 
in a 5-week training class. Topics include sanctuary and park resource protection 
programs, interpretation techniques, and an overview of the physical, biological, and 
cultural aspects of the Santa Barbara Channel and the Channel Islands. 

Results 

• Engagement -137 volunteers actively participate in the program annually. The 
program has a high level of retention and represents community members with 
varied backgrounds from scientists to teachers and students to the retired. They 
donate approximately 34,000 hours; 
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• Reach - Volunteers reach over 400,000 people annually through participation in 
over 35 outreach events in Ventura and Santa Barbara and Los Angeles counties 
and as naturalists aboard 8 whale watch vessels in 3 main harbors; 

• Budget - All undertakings by CINMS and CINP are subject to the availability of 
appropriated funds. CINMS annual volunteer program budget, not including staff 
salaries, is 10 thousand dollars. A percentage of the CINP volunteer fund is 
provided each year to the program. This amount varies depending on the needs 
each year of additional Park volunteer programs. In addition, CINP will provide 
in-kind staff services and CINP visitor center auditorium facility for training classes 
and meetings. 

• Supervision - Personnel facilitate and coordinate whale watch calendar with all 
non-concessionaire and CINP concessionaire non-landing trips. NPS coordinates 
directly with CINP concessionaires to coordinate CINC “Island Hike Naturalist” 
staffing on both day and overnight landing trips to the islands; 

• Scheduling - CINMS and CINP coordinate staffing at community events. Efforts will 
be made to have both agency booths next to each other at events in order to share 
volunteer staff and to demonstrate the agency partnership; 

• Liability - CINC volunteers work under an interagency position description and 
sign both agencies’ volunteer agreements and liability forms. Liability for 
non-landing trips is covered by CINMS. Liability for all landing trips, where 
volunteers go ashore to lead a hike, is covered by CINP. Vessel operators purchase 
and maintain comprehensive general liability insurance and such insurance 
policies shall be assumed by, credited to the account of, and undertaken at vessel 
operators’ sole risk; and 

• Recognition - CINMS has a volunteer tracking database and recognizes volunteers 
for efforts at an annual volunteer recognition dinner. CINP helps with this event 
and provides “Volunteer Recognition” day trips to the islands throughout the year. 
Program also has national prestige and has received the US Department of 
Commerce’s Take Pride in America national award for the best Federal volunteer 
program in 2011. A CINC volunteer received the 2012-13 National Marine 
Sanctuary Foundation Volunteer of the Year award. 

Lessons Learned 

It is important to diversify funding streams (appropriated and through foundations) to 
maintain a fully operational volunteer program like the Channel Islands Naturalist Corps. 
The return on investment with a volunteer program like this is remarkable, 135 
volunteers donate approximately 34 thousand hours and reach over 400 thousand people 
annually. However with declining federal budgets, it is becoming necessary to seek 
outside funding and sponsorship sources to maintain all the vital functions of the program 
(i.e. uniforms, recognition dinner, training, staffing). 

Current Status of the Partnership 

CINMS is still obligating program funds toward CINC supplies and recognition budget; 
however, both agencies struggle to provide adequate staffing to maintain such a large 
volunteer program. 
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Conclusion 

The partnership enables CINMS and CINP to have more efficient use of community 
volunteers and allow each agency to expand its educational services while also providing 
additional opportunities for volunteers. 
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DISCUSSION 


Evaluation activities were conducted to clarify the definition of NOAA Education 
high-return partnerships, identify partnership activities, and identify case studies to 
understand the mechanisms that operate within a high-return partnership. 

Research Questions 

This section of the report summarizes the findings based on the research questions and 
data collected. 

Research Question 1: How can NOAA partners help meet our mission? 

Partners can help complete the NOAA mission by extending the reach of programs, and by 
increasing program efficiency and effectiveness. These outcomes cannot be achieved by 
NOAA alone, and therefore partners are critical in meeting NOAA’s mission. Each outcome 
is discussed below, supported by results from survey data and case study analyses. The 
logic model (Figure 18) illustrates how partners and partnerships support the NOAA 
mission. 

Figure 18. NOAA Education Partnerships Logic Model. 


Reach 

NOAA cannot hope to engage the entire Nation in terms of education and outreach. 
Partnerships expand NOAA’s reach, leveraging partners’ own relationships with audiences. 
Understanding the audience of the partner and the kind of relationship they have with 
them is an important part of partnership formation. The NOAA Education Partnership 
Survey data and selected case studies illustrate NOAA’s expanded reach to audiences on 
very large scale. For example: 

• Respondents indicated that new audiences were reached by 97% of partnerships 
[Analysis Question 11]; 

• In 2012, the Exploratorium [Case Study 1A] reached 570 thousand visitors living in 
or visiting the San Francisco area ; 
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• The Science on a Sphere® [Case Study 1C] has more than 100 installations 
worldwide; 

• The AMS partnership [Case Study 2A] has reached more than 140 thousand K-12 
teachers and 4.9 million students directly or indirectly since the inception of the 
program in 1990; and 

• Volunteers reach more than 400 thousand people annually through the CINMS and 
CINP partnership [Case Study 3B], 

Simply reaching more people is not enough, partner institutions should also share the 
NOAA Education mission and goals. Results from the NOAA Education Partnership Survey 
and resulting case studies indicate how partners share the NOAA mission: 

• Respondents indicated that they include opportunities for NOAA messaging (84%), 
help NOAA address education strategic goals and objectives (79%), and share 
similar values and qualities (78%) [Analysis Question 14]; and 

• The EPA GLNPO and CGLL [Case Study 3A] partnership allows Great Lakes Sea 
Grant Educators to achieve all six outcomes under NOAA’s educational goal of 
advancing environmental literacy. 

A heretofore unacknowledged benefit is that an education partnership can move beyond 
education to extend NOAA’s operational reach, at least in the case of Exploratorium [Case 
Study Example 1A], This partner has now become part of the observing system and is a 
port where NOAA ships can dock. This partnership contributes to the NOAA mission on 
education, operations and science arms of the agency. 

Reach can also be considered geographically. NOAA’s mission covers the entire United 
States, U.S. territories, its oceans and coasts, and the mission differs in each region. Using 
maps such as Figure 8, NOAA can assess gaps in partnerships to reach all regions, for 
example a state in a region is not represented by a partner. Leveraging regional 
investments and partnering with other groups that share regional missions allows NOAA to 
reach people in the region focused on that region’s issues. For example: 

• Respondents indicated that the majority (73%) of NOAA education partnerships 
occur on a regional or smaller scale. Specifically, 40% were reported as local, 21% 
as state and 12% as regional [Analysis Question 6]; 

• Respondents indicated that partnerships take place in local communities where 
NOAA offices, labs, sanctuaries, and reserves are located (64%) [Analysis Question 
14] as shown in Figure 8; 

• The CINMS and CINP partnership [Case Study 3B] focuses on educating people of all 
ages about the park and sanctuary’s special terrestrial and marine protected area 
of national and global significance through a partnership with vessel operators 
and the training a cadre of volunteers. 

Efficiency 

NOAA must make the most its resources, funding, and staff. NOAA Education cannot expect 
additional resources in order to reach and educate the Nation in our mission. Partnerships 
increase efficiency of allocated resources. By combining forces with partners that may also 
be resource-limited, NOAA and its partners can work together on a shared mission. The 
characteristics of NOAA and the partner must be complementary and symbiotic. Authority 
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and organizational characteristics can be critical to success when they complement each 
other. For example: 

• The Science on a Sphere® [Case Study 1C] partnership, with more than 100 
collaborating institutions worldwide, provides effective and efficient 
communication of NOAA-related science and messaging to the public, reaching 
more than 35 million visitors annually; and 

• Great Lakes partners [Case Study 3A] share and contribute resources to run 
programs collaboratively. 

Effectiveness 

NOAA is not an expert in everything and cannot expect to hire and bring experts for 
everything it needs to accomplish. Partnerships improve program effectiveness by 
leveraging a partner’s expertise. The core business of a partner might be to develop 
interactive science exhibits. This will never be the core business of a federal agency, yet 
NOAA can partner with others and bring its expertise in science content, which is NOAA’s 
core business. Leveraging the strengths of each partner should result in a more effective 
program, experience or engagement with the target audience than if each partner were to 
go it alone. 

Research Question 2: How does NOAA maximize NOAA Education partnerships? 

Maximizing partnerships can occur in a variety of ways, by providing funding and staff 
time, leveraging resources, increasing reach or maximizing personal relationships (i.e., the 
“people factor”). Not all partnerships bring in funding, but many partnerships provide 
reach beyond what could be accomplished in the absence of the partnership. A clear and 
significant finding of this study is the important role that human and social capital plays in 
maximizing NOAA Education high-return partnerships. This is echoed by the NRC (2010) 
report which noted that “people are NOAA’s most valuable assets” (p. 149). Another key 
finding of this study is the importance of the maturity of the partnership. While not 
necessarily merely measurable in years, a mature NOAA Education high-return 
partnership provides significant returns, often with minimal but necessary maintenance. 

Funding and staff time 

NOAA must understand its contributions in terms of funding and staff time regarding 
high-return partnerships. The NOAA Education Partnership survey asked respondents to 
estimate NOAA’s contribution in the form of money invested, the staff time in the form of 
FTE, and what NOAA and the partner contributed in the form of resources. 

• Not all partnerships require a great deal of funding or staff time [Analysis Question 
7 and 8], 

Although beyond the scope of this study, the PWG wondered if the monetized outcomes or 
benefits of the partnership was greater than the monetized inputs or costs. Measurement 
of monetary partnership contributions can be achieved through a cost-benefit analysis. 
NOAA Education is now well positioned to conduct such an analysis and monetize the 
contributions and outcomes of a partnership. The work will be extensive, and require a 
sense of the type of outcomes to be monetized. Only a few NOAA Education partnerships 
will have sufficient information required for a cost-benefit analysis. 
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Leveraging Resources, Increasing Reach, and Maximizing Personal Relationships 

Results from the NOAA Education Partnership Survey and resulting case studies provide 

examples of the contributions and outcomes as a result of partnerships. For example: 

• Respondents indicated that new audiences were reached by 97% of partnerships 
[Analysis Question 11]; 

• Local and in-kind funding and resources are leveraged in the Great Lakes [Case 
Study 3A: EPA GLNPO and CGLL] to facilitate the Great Lakes Literacy effort, which 
is strongly supported by connections between people committed to sustaining the 
collaborative efforts; 

• Local resources and expertise are leveraged [Case Study 3B: CINP]; 

• Personal relationships are leveraged as well as dollars [Case Study 1A: 
Exploratorium; Case Study IB: OMCA; Case Study 1C: SOS; Case Study 3A: EPA 
GLNPO and CGLL]; 

• NOAA offices are able to expand upon a partnership to involve broader 
participation, including the science arm of the agency [Case Study 1A: 
Exploratorium; Case Study 2A: AMS]; 

• Partners leverage NOAA funding and support into additional investments from 
other funders [Case Study 2A: AMS; Case Study 3B: CINP]; and 

• Benefits as an indirect result of partnership (e.g., sustained capacity, investment in 
infrastructure, permanent exhibit leads to programming) [Case Study 1A: 
Exploratorium; Case Study IB: OMCA], 

Another consideration of how NOAA maximizes Education partnerships is to examine 
non-monetary quantitative data in terms of touchpoints. For example: 

• The number of parts of NOAA involved in the partnership; 

• The number of parts of the partner organization involved in the partnership; 

• The number of activities completed as a result of the partnership; and 

• The number of resources (or assets) shared between the partners beyond normal 
duties (e.g. buoy data, NOAA equipment, scientific expertise) 

Level of Maturity of the Partnership: 

Partnerships may reach a level of maturity where relatively modest in-kind contributions 
by NOAA are needed to sustain the partnership [Case Study Example 1A: Exploratorium; 
Case Study Example 2A: AMS], At this stage, significant returns often result. However, the 
majority (76%) of NOAA Education partnerships reported in this study have been in place 
for eleven years or less [Analysis Question 4], While the maturity of a partnership cannot 
be calculated simply be the number of years in existence, partnerships that end “too early” 
may not reach the high-return stage. It is in NOAA’s best interest to facilitate and nurture 
the most promising partnerships to attain a high level of maturity to fully realize its 
potential and maximize the benefits to all parties involved. 

Research Question 3: What qualities are shared by NOAA Education high-return 
partnerships? 

In an effort to focus the quality and number of partnerships reported in the NOAA 
Education Partnership Survey, respondents were asked to select their “top 5” high-return 
partnerships. High-return was defined by the PWG as “benefits to NOAA and partners 
outweigh the investment and involvement for all parties involved.” The PWG assumed all 
partnerships reported were indeed high-return, and cross-case study analysis revealed 
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several aspects as common and integral components of a NOAA Education high-return 
partnership. 

Several characteristics are common among NOAA Education high-return partnerships. The 
case studies illustrate many of these characteristics, and commonalities were further 
categorized into performance or organizational aspects. 

Performance aspects, or how the partnership is formed, staffed, and extended, indicative 
of a high-return partnership include: 

• Partners leverage NOAA dollars to secure funding and in-kind support from other 
sources [Case Study 1C: SOS; Case Study 3A: EPA GLNPO and CGLL]; 

• Staff time invested by partners is equal to or greater than NOAA staff time [Case 
Study 1C: SOS]; 

• Partnership as networks a large number of multiple partners [Case Study 1C: SOS; 
Case Study 3A: EPA GLNPO and CGLL]; 

• Partnership opens the doors to new partners [Case Study IB: OMCA; Case Study 1C: 
SOS; Case Study 2A: AMS; Case Study 2B: PIN; Case Study 3A: EPA GLNPO and 
CGLL]; 

• NOAA resources are used in new and different ways [Case Study 1A: 

Exploratorium; Case Study 1C: SOS; Case Study 2B: PIN]; and 

• Opportunities are capitalized on as they arise [Case Study 2B: PIN; Case Study 3A: 
EPA GLNPO and CGLL; Case Study 3B: CINP], 

Organizational aspects, or how the partners work together and manage the partnership, 
indicative of a high-return partnership include: 

• Successfully forging gaps between different organizations (i.e., NOAA and partner) 
[Case Study 1A: Exploratorium]; 

• Partner learns about NOAA and the capabilities of the agency [Case Study 1A: 
Exploratorium; Case Study 1C: SOS; Case Study 2A: AMS]; 

• Partner successfully navigates NOAA bureaucracy and increases connectivity to 
different parts of NOAA [Case Study 1A: Exploratorium; Case Study 2A: AMS]; and 

• Communication occurs through primary representatives from each partner [Case 
Study 1A: Exploratorium; Case Study 1C: SOS; Case Study IB: OMCA; Case Study 2A: 
AMS], 

Research Question 4: What tools and strategies are needed for an effective NOAA 
Education partnership portfolio? 

A variety of tools and strategies are necessary for an effective partnership portfolio. An 
NRC report on learning in informal settings (2009) outlined effective partnership strategies, 
including “identifying shared goals; designing experiences around issues of local relevance; 
supporting participants’ patterns of participation (e.g., family structure, modes of 
discourse); and designing experiences that satisfy the values and norms and reflect the 
practices of all partners” (p.301). While specified for informal environments, this study 
found the strategies applicable across NOAA high-return Education partnerships. 
Additional NOAA Education partnership-specific strategies were also identified based on 
this study, including: 

NOAA Education partnerships are most effective when there is: 
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• Regular, ongoing communication between NOAA and the partner(s); 

• An emphasis on meaningful evaluation; 

• Support from both NOAA and partner leadership; and 

• A cultivation of critical personal relationships. 

When forming a new or transitioning an existing NOAA Education partnership, the 
following should be reviewed: 

• Partnership definition and criteria; 

• Partnership mechanisms for both formation and dissolution; and 

• Programming goals and resources. 

The following are essential for an effective NOAA Education partnership portfolio: 

• Involving NOAA General Counsel when providing resources or support beyond 
normal duties; 

• Determining whether a MOU or MOA is the best to use and knowing that 
mechanisms can change over time; and 

• Establishing a point of contact in each NOAA line office to facilitate partnership 
formation. 


Response to NRC Recommendations 

Part of the context and rationale of this study was to address the recommendations from 
the National Research Council’s (NRC) NOAA’s Education Program: Review and Critique 
(2010) to better understand how NOAA Education partnerships are formed, fostered, 
sustained, and evaluated. As the report notes, “[plartnerships will be critical if NOAA is to 
reach the ambitious goals identified in the strategic plan, because the agency does not have 
the resources to achieve its goals on its own” (p.60). Accordingly, a stated goal of this 
report was to provide information and recommendations to the NOAA Education Council 
and NOAA Education community to use in making strategic decisions regarding NOAA 
Education partnerships. 

The NRC report covered an array of topics regarding NOAA Education. A subset of 
recommendations relevant to partnerships were selected from the report subtopics of 
NOAA’s role in education, education goals and outcomes, and education evaluation 
practices. The pertinent recommendations are outlined below, along with a response based 
on information collected and analyzed for this study. 

NRC Recommendation 1.1 

NOAA should fulfill Us role in education through the use of...partnerships with local and 
state education infrastructure, academic institutions, government agencies, business and 
industry, and private-sector and nonprofit organizations; and the agency’s global science and 
international partnerships. 

The results of this study indicate that NOAA Education staff have been and continue to 
forge partnerships with a variety of local, state, regional and national institutions and 
agencies. Sixty-seven partnerships were identified in this study, some spanning greater 
than 20 years. 
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As noted in the NRC report (2010), “the resources [NOAA] manages provide vast and 
important education opportunities, and management of of these environments provides 
the agency with connections to the surrounding communities and organizations” 
(Conclusion 1.4, p.135). Indeed, the NRC report further articulates “NOAA’s ability to make 
good use of its education resources and assets to engage a substantial number of education 
activities is impressive. Most partnerships with educational organizations, other agencies, 
and institutions with complementary STEM expertise have enhanced the reach and impact 
of NOAA’s education efforts. These partnerships have often yielded additional expertise, 
educational tools, mechanisms for dissemination, and matching funds” (p. 148). Nearly 
every case study outlined in this study provided specific examples of such partnerships. 
The additional data from this study can support implementation of this recommendation, 
as NOAA Education now has a road map to foster and facilitate efficient and effective 
education partnerships across line offices and on multiple scales (i.e., local, state, regional, 
federal and international). 

NRC Recommendation L3: 

Within the constraints of NOAA’s mandates in education, the agency should continually 
evaluate where it leads, collaborates, follows or declines to participate in partnerships with 
others. These decisions should be guided by consideration of the agency’s role, assets, 
resources, and priorities in education and the strengths and missions of other agencies, 
institutions, and organizations engaged in education. 

As candidly recognized by the NRC Committee, “the Education Council does not have 
budgetary or institutional control over the education efforts of NOAA line and program 
offices”, and several have individual education mandates within their legislation, often 
with “local components with local control” (p.143). 

Because there is no common data collection and the NOAA Education partnership portfolio 
is not under one authority, it is difficult to shape the portfolio as recommended. This study 
is the first attempt to examining the NOAA Education Partnership portfolio. The working 
group’s recommendations in this study are based on how the NOAA Education community 
operates, and therefore include providing training and guidance as needed and to 
individuals. 

NOAA Education can focus on assuring that partnerships have clear, mutually agreed upon 
goals, make wise use of NOAA resources, and provide a reach that NOAA would not have in 
the absence of the partnership. Additionally, the NRC Committee (2010) concluded that 
“NOAA can play a supporting role in state and local education” and should “focus on 
productive partnerships to support local and state education systems while promoting 
NOAA’s education and stewardship mission.” Furthermore, the report notes “education 
efforts are more likely to be productive if they align with national and local education 
needs” and “make better use of assets and programs that already exist” (Conclusion 1.7, p. 
136). Data from this study support these conclusions, and that NOAA Education has case 
studies as examples from which to draw to further implement this recommendation on a 
larger scale. 

NRC Recommendation II.2 

To reach NOAA’s environmental literacy goal, the Education Council should develop its 
implementation plan and future revisions of the education strategic plan to... articulate how 
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NOAA education programs will draw on the scientific, engineering, research, and other 
expertise accessible within the agency as well as in the broader community. 

A component of this recommendation focuses on partnerships internal to NOAA and was 
considered beyond the scope of this study. However, data from NOAA Education 
Partnership Survey and case studies included in this report would provide an excellent 
foundation upon which to further explore these important contributions as related to the 
broader community (i.e., Analysis Question 13 and several case studies). 

NRC Recommendation IV.l 

The Education Council should continue to improve the evaluation expertise of its education 
program managers, contract with external evaluators for summative evaluation, and require 
the incorporation of the most appropriate and rigorous evaluation strategies during program 
development to guide design, continual improvement, and delivery of its education programs. 

While not directly related to the scope of this study, improving the evaluation expertise of 
members of the PWG was a goal for the Evaluation leads. In developing the collaborative 
effort between program staff and evaluators to collect data regarding NOAA Education 
partnerships, members of the PWG were tasked with developing research question(s), logic 
model, scope of the study, data inventory, data collection and analysis, and actionable 
recommendations for the NOAA Education Council. While this was a small step toward 
improving evaluation expertise among program staff, the experience provided a 
foundation for developing an understanding of the complexities and intricacies involved in 
the evaluation process. The Evaluation leads believe this will assist members of the PWG in 
understanding what is required to better evaluate, design, improve and deliver NOAA 
Education programs. 

NRC Recommendation IV.4 

Education programs should evaluate internal collaboration among line offices and between 
education and operational and scientific staff, as well as the quality of external partnerships 
with other agencies, institutions, organizations, and the broader STEM communities. 

As stated previously, partnerships internal to NOAA were considered beyond the scope of 
the Portfolio Review study. In terms of examining the quality of external partnerships, the 
PWG directly addressed this component of the recommendation. Accordingly, a 
recommendation of the PWG will be included to address this via annual completion of the 
NOAA Education Partnership Survey by NOAA educators. 
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RECOMMENDATIONS 


Actionable recommendations are the most important part of the Portfolio Review process 
and drove the inquiry of this study from the start. The recommendations developed by the 
PWG are intended to be actionable by the NOAA Education Council. Education Council 
should take on the following tasks: 

PWG Recommendation 1. Develop a guidance manual for establishing new 
partnerships or transitioning existing ones. 

The audience for this guidance manual would be NOAA educators both in the field and at 
the headquarters level. The guidance would provide checklists of items that should be 
addressed, as well as procedural instructions for those tasks that need to be completed 
step-by-step. The manual would provide general guidance regarding key questions that 
should be considered. Use the partnership definition and criteria as a starting point and 
this report for case studies for the manual. 

PWG Recommendation 2. Disseminate the guidance manual and a collection of MOUs 
as resources for partnership development. 

The working group should consider how such information and guidance would best 
support NOAA educators seeking partnerships. Making the resources available on a 
website will be important part of the dissemination strategy. 

PWG Recommendation 3. Provide training for NOAA educators in forming new 
partnerships and transitioning existing partnerships. 

Although providing a one-time webinar will likely be ineffective, just-in-time and 
context-appropriate support at the right time would make this guidance most effective. 
Additionally, general education of all NOAA Educators would make them aware of when 
they should seek this information. 

PWG Recommendation 4. Conduct a cost-benefit analysis of partnerships. 

As noted in the Discussion section, conducting a cost-benefit analysis will more definitively 
answer questions about how partnerships are maximized. 

PWG Recommendation 5. Conduct a gap analysis based on the existing dataset of 
NOAA Education partnerships. 

The authors of this report recognize that there are gaps in the data collected, as not all of 
the NOAA Education community reported partnership data into the survey. In the future, a 
review comparing the survey results with the entire NOAA Education community could 
identify gaps. Questions and figures used in this study could serve as a guide to this 
analysis. For example, identifying geographic gaps by comparing partners locations with 
NOAA locations as shown in Figure 17. 

The Education Council’s Monitoring and Evaluation Working Group should take on the 
following tasks: 

PWG Recommendation 6. Develop partnership performance measures, track the total 
number of partnerships, goals, money, leverage and who is involved. 

The M&E Working Group should consider taking on the task of monitoring partnerships on 
an annual basis. Measures would be developed for Council member to report into on an 
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annual basis. This would include the total number of partnerships, partnership goals, the 
level of funding and personnel involved. 

PWG Recommendation 7. Review and revise the NOAA Education Partnership Survey. 

If the M&E Working Group takes on the task of monitoring partnerships annually, the 
NOAA Education Partnership Survey can be adapted as the primary data collection 
instrument. Additionally, if a gap analysis is conducted and data is collected regarding 
partnerships not previously reported, the Survey can be improved to better collect data to 
align with the analysis. Working groups can propose changes to Survey to better assess the 
return on partnership. 

PWG Recommendation 8. Explore the role that partnerships contribute to existing 
performance measures. 

As displayed in the results of Analysis Question 9, partners share the NOAA mission and 
performance measures in terms of reaching similar audiences, educators, K-12 students 
and postsecondary students. The M&E Working Group should consider collecting the 
contribution of partners in helping NOAA meet our mission and how partners contribute 
to our common performance measures to demonstrate a collective impact. Communication 
between NOAA and partners will help determine the documentation process so both 
entities receive proper credit for their efforts. 
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APPENDICES 


Appendix A: NOAA Education Partnership Survey 

High-Return Education Partnerships 

The new Partnerships Working Group of the Education Council is completing the work of the former Pre-Kindergarten 
through Early Career Working Group (PK-20+ Working Group) and Finding Innovative Ways to Connect NOAA to the 
Public (Connects). We are requesting information from NOAA Program and Line Offices regarding intra-agency; 
interagency; and external (non-profit, university, corporate) partnerships. The request is in direct response to the 
recent (2010) National Research Council (NRC’s) recommendation for NOAA Education to establish and coordinate 
partnerships with organizations whose purposes are to conduct or support science education. It is also important as 
part of NOAA's Education Strategic Plan 2009-2029, Outcome 1.5, NOAA works cooperatively to maximize the 
impact of federal investment in ocean, coastal, Great Lakes, weather, and climate education. 

For purposes of this survey, we are requesting that you report up to 5 important partnerships that your 
programs/projects have through grants, cooperative agreements, contracts, or that use in-kind mechanisms 
(MOU/MOA) and have significant benefits to NOAA and advance NOAA's Education Strategic Plan. 

Once collected, the survey will provide a snapshot of NOAA’s reach to stakeholders and will help to focus future 
efforts in areas that are lacking. 

If you have questions, please do not hesitate to contact the Partnerships Working Group: Kate Thompson or Frank 
Niepold at kate.thompson@noaa.gov or frank.niepold@noaa.gov. Thank you for your cooperation. We look forward 
to your participation in this survey. 


Education Council Administrative Unit: * 

Instruction: From the drop down menu, select the program or office for which you are reporting. 



Definition: Partnership 

A partnership is an agreement between two or more organizations, created to achieve or assist in reaching a 
common goal. Partnerships may involve one organization utilizing another's unique abilities, equipment or services, or 
it may be a "sharing" of resources (money, time, knowledge, equipment, etc.) to accomplish shared objectives 
between the participating partners. (3/27/13) 


Name of Partner * 

Instruction: In the text box, report the name of the informal education institution that hosted a NOAA exhibit or docent 
training program in FY12. See above for more detailed instructions on what counts. 
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Partnership Description * 

Describe the partnership, and type(s) of activities. 


When did the Partnership begin? 

Calendar Year (e.g. 2007) 


Location of Partner 

City, State and/or Zip 


Type of Institution of Partner* 

O K-12 Institution 
O College/University 
O Federal Agency 
O State Agency 
O Local Agency 
O Corporation 

O Zoos, Aquariums and Museums 
O Science Research Institutions 
O Non-Profit/NGOs (policy, professional orgs) 
O Other:_ 


Partnership Mechanism * 

□ Grant 

□ Cooperative Agreement 

□ MOU/MOA 

□ Donations to the Department 

□ Statutory Partnership 

□ None 

□ Other:_ 


Current Program Status 

O Active and fully operational (76% to 100%) 
O Partially operating (51% to 75%) 

O Minimally operating (1 to 50%) 

O Not active (0%) 

O Other:_ 


How much funding has NOAA contributed to the partnership? * 

Total contribution in US dollars (includes monetary value of staff time, other in-kind contributions, cash). Excludes 
funding for non-educational activities. Range of dollars or descriptions of funding if cannot calculate. 
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What is the staff time for NOAA (in FTE) of maintaining this partnership? 


Page 2 


Partnership Assessment 

This section is meant to provide insight into the processes surrounding the rationale and methodology for selecting 
partners, sustaining partnerships, and terminating partnerships. 


Partnership alignment with NOAA's current educational output measures 

□ Formal education programs 

□ Informal education programs 

□ Professional development programs 

□ Higher education programs 


What NOAA provides in Technical and Knowledge Transfer 

in-kind contributions 

□ Instruments/ lab equipment 

□ Materials (curriculum, activity kits, literacy guides, outreach materials) 

□ Expertise (scientific, educational) 

□ Real estate (for interpretive signs) 

□ Facilities (places for programs) 

□ Exhibit space 

□ Datasets and Data visualizations 

□ Exhibit development 

□ Other:_ 

Please, elaborate on the above criteria that you selected for what NOAA provides. 


What the Partner Provides in Technical and Knowledge Transfer 

in-kind contributions 

□ Instruments/ lab equipment 

□ Materials (curriculum, activity kits, literacy guides, outreach materials) 

□ Expertise (scientific, educational) 

□ Real estate (for interpretive signs) 

□ Facilities (places for programs) 

□ Exhibit space 

□ Datasets and Data visualizations 

□ Exhibit development 

□ Other:_ 
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Please, elaborate on the above criteria that you selected for what the Partner provides. 


What were the SELECTION criteria for selecting the partner? 

Select all that apply. 

□ Partner has established relationships that will help provide a different level of POLICY access to those networks 
for NOAA Education officials 

□ Partner has established relationships that will help provide a different level of SCIENCE CONTENT KNOWLEDGE 
(technical expertise) to those networks for NOAA Education officials 

□ Partner has established relationships that will help NOAA provide a different level of access to new or refined 
approaches for AUDIENCES/CONSTITUENTS/STAKEHOLDERS 

□ Partner can contribute to NOAA’s WORKFORCE DEVELOPMENT goals (AAAS Fellows, Einstein Fellows, 
Presidential Management Fellows, Minority Serving Institutions, etc.) 

□ Partner is selected through competition 

□ Other:_ 


What mechanisms were used in determining if the partner met the SELECTION criteria? 

Select all that apply. 

□ Conduct competition to select the partner (or Apply series of criteria or a rubric to choose the partner) 

□ Conduct background research on proposed partner (to ensure partner stability, suitability) 

□ Host one or more meetings with proposed partner prior to entering into formal agreements 

□ Conduct an onsite partner visit to observe, examine, and analyze organizational structure and program efficacy 

□ Consider asking other NOAA offices to join with your Line Office in the proposed partnership 

□ Inquire of potential partner if there have been (or are) other existing (or past) NOAA partnerships 

□ Adapt existing MOAor MOU to fit this partnership 

□ Other:_ 


Please, assess your partnership based on the following criteria. 

Select all that apply 

□ The partner shares similar values and qualities to those of NOAA. Our corporate values are Science, 

Stewardship and Service. Our agency is successful, welcoming, trusted, innovative, reliable, customer-focused, and 
collaborative. 

□ The partnership creates stories or case-studies that will be relevant, interesting and intriguing for NOAA 
stakeholders directly as a result of operating unit(s) involvement. 

□ The partnership provides opportunities for NOAA involvement by either our staff or our constituents. 

□ The partnership provides a high return to NOAA's local communities where NOAA offices/labs/sanctuaries and 
reserves are located (high return means benefits to NOAA and partners outweigh the investment and involvement for 
all parties involved). 

□ The partnership provides a high return to NOAA staff and our constituents (high return means benefits to NOAA 
and partners outweigh the investment and involvement for all parties involved). 

□ The partnership provides a high return to International/National/regional/local initiatives (high return means 
benefits to NOAA and partners outweigh the investment and involvement for all parties involved). 

□ The partner organization has broad connections with communities of influence that would further our mission and 
goals. 

□ The partnership includes opportunities for NOAA messaging (ex. conservation, stewardship, science, 
awareness, safety and preparedness). 

□ The partnership is designed to make a lasting contribution to NOAA communities. 

□ The partner is helping us address our education strategic goal/objectives. 

□ Other:_ 


65 




Please, elaborate on the ASSESSMENT criteria that you selected above. 


Page 3- 


Final Page 

Instructions: Before clicking SUBMIT, review your reported data by clicking on the BACK button to review the your 
submission on the previous pages. 

OPTIONAL: Provide information about how data was collected, assumptions that were used, or other 
remarks about the data submitted. 

This data is used to understand the context of what was reported. 
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Appendix B. List of Acronyms 


AERA 

AMS 

ASTC 

B-WET 

CENCOOS 

CGLL 

CINMS 

CINP 

COSEE 

CPC 

CPESE 

CPO 

DOC 

ELG 

EPA 

ESRL 

FEMA 

FTE 

GAO 

GLNPO 

GLRI 

IAGLR 

ILI 

ISE 

LELP 

LIT 

M&E WG 

MOA 

MOU 

NAPA 

NASA 

NCDC 

NCEP 

NESDIS 

NGO 

NMFS 

NOAA 

NOS 

NPS 

NRC 

NSF 

NSOF 

NWS 

OAR 

OED/OEd 

OMCA 

ONMS 


Atmospheric Education Resource Agents 

American Meteorological Society 

Association for Science and Technology Centers 

Bay Watershed Education and Training 

Central and Northern California Ocean Observing System 

Center for Great Lakes Literacy 

Channel Islands National Marine Sanctuary 

Channel Islands National Park 

Centers for Ocean Sciences Education Excellence 

Climate Prediction Center 

Cooperative Program for Earth System Education 

Climate Program Office 

Department of Commerce 

Environmental Literacy Grant 

Environmental Protection Agency 

Earth System Research Laboratory 

Federal Emergency Management Agency 

Full Time Equivalent 

Government Accountability Office 

Great Lakes National Program Office 

Great Lakes Restoration Initiative 

International Association for Great Lakes Research 

Institute for Learning Innovation 

Informal Science Education 

Lake Erie Environmental Literacy Principles 

Local implementation Team 

Monitoring & Evaluation Working Group 

Memorandum of Agreement 

Memorandum of Understanding 

National Academy of Public Administration 

National Aeronautics and Space Administration 

National Climatic Data Center 

National Centers for Environmental Prediction 

National Environmental Satellite, Data, and Information Service 

Non-Governmental Organization 

National Marine Fisheries Service 

National Oceanic and Atmospheric Administration 

National Ocean Service 

National Parks Service 

National Research Council 

National Science Foundation 

NOAA Satellite Operations Facility 

National Weather Service 

Office of Oceanic and Atmospheric Research 

Office of Education 

Oakland Museum of California 

Office of National Marine Sanctuaries 
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OPM 

Office of Personnel Management 

PIN 

PLAN IT NOW 

PK-20 WG 

Pre-Kindergarten through Early Career Working Group 

PORTS 

Physical Oceanographic Real-Time Systems 

PPR 

Partnerships Portfolio Review 

PWG 

Partnerships Working Group 

SG 

Sea Grant 

SOS 

Science on a Sphere® 

STEM 

Science, Technology, Engineering and Mathematics 

USFWS 

US Fish and Wildlife Service 

WG 

Working Group 

WRN 

Weather Ready Nation 

YMP 

Young Meteorologist Program 

ZAM 

Zoos, Aquariums and Museums 
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